You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Nisha Menon <ni...@gmail.com> on 2018/02/16 08:16:51 UTC
Fwd: KNOX Pac4j Azure AD Open ID
Hello,
I have setup KNOX to connect with Azure AD using pac4j.
However, after the authentication at Azure login page, it gets into an
infinite loop and does not give back the original REST call response.
*Details:*
1. I try to access the original URL eg:
*https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
<https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
2. It redirects to *https://**login.microsoftonline.com
<http://login.microsoftonline.com/>* and asks for credentials.
3. After successful login at Azure login page, it redirects to
*http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
<http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session and
state variables passed as below:
*https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
<https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
2. Following this call, it *again *calls the *login.microsoftonline.com
<http://login.microsoftonline.com/>* like below:
*https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
<https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
After this, step 1 and 2 alternate several times and finally lands up in
"ERR_TOO_MANY_REDIRECTS"!!!
This is my knoxsso.xml:
1. <topology>
2. <gateway>
3. <provider>
4. <role>webappsec</role>
5. <name>WebAppSec</name>
6. <enabled>true</enabled>
7.
<param><name>xframe.options.enabled</name><value>true</value></param>
8. </provider>
9. <provider>
10. <role>federation</role>
11. <name>pac4j</name>
12. <enabled>true</enabled>
13. <param>
14. <name>pac4j.callbackUrl</name>
15.
<value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
16. </param>
17. <param>
18. <name>clientName</name>
19. <value>OidcClient</value>
20. </param>
21. <param>
22. <name>oidc.id</name>
23. <value>385c2bc*****************2695eaa34</value>
24. </param>
25. <param>
26. <name>oidc.secret</name>
27.
<value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
28. </param>
29. <param>
30. <name>oidc.discoveryUri</name>
31.
<value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
32. </param>
33. </provider>
34. <provider>
35. <role>identity-assertion</role>
36. <name>Default</name>
37. <enabled>true</enabled>
38. </provider>
39. </gateway>
40. <application>
41. <name>knoxauth</name>
42. </application>
43. <service>
44. <role>KNOXSSO</role>
45. <param>
46. <name>knoxsso.cookie.secure.only</name>
47. <value>false</value>
48. </param>
49. <param>
50. <name>knoxsso.token.ttl</name>
51. <value>30000</value>
52. </param>
53. <param>
54. <name>knoxsso.redirect.whitelist.regex</name>
55.
<value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
56. </param>
57. </service>
58. </topology>
I tried using response_type "id_token", enabling nonces, knoxsso.secure to
true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
gateway-audit.log when redirection error starts:
1. 18/02/15 12:38:02
||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response
status: 302
It clearly shows Response status as "302" and not "200". This leads to
redirection!
What could I be missing here? Any pointers will be greatly appreciated.
Regards
Nisha
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hello Sandeep,
Sorry, I had forgotten to share the topology.
*myCluster.topology:*
1. <topology>
2. <gateway>
3. <provider>
4. <role>webappsec</role>
5. <name>WebAppSec</name>
6. <enabled>true</enabled>
7. <param>
<name>cors.enabled</name>
<value>true</value>
</param>
8. </provider>
9. <provider>
10. <role>federation</role>
11. <name>SSOCookieProvider</name>
<enabled>true</enabled>
<param>
<name>sso.authentication.provider.url</name>
<value>https://52.169.143.168:8442/gateway/knoxsso/api/v1/websso</value>
</param>
12. </provider>
13. <provider>
14. <role>identity-assertion</role>
15. <name>Default</name>
16. <enabled>true</enabled>
17. </provider>
18. </gateway>
19. <service>
<role>NAMENODE</role>
<url>hdfs://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:8020</url>
</service>
<service>
<role>JOBTRACKER</role>
<url>rpc://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:8050</url>
</service>
<service>
<role>WEBHDFS</role>
<url>http://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:50070/webhdfs</url>
</service>
20.
21. <application>
22. <name>knoxauth</name>
23. </application>
24. <service>
25. <role>KNOXSSO</role>
26. <param>
27. <name>knoxsso.cookie.secure.only</name>
28. <value>false</value>
29. </param>
30. <param>
31. <name>knoxsso.token.ttl</name>
32. <value>30000</value>
33. </param>
34. <param>
35. <name>knoxsso.redirect.whitelist.regex</name>
36.
<value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
37. </param>
38. </service>
39. </topology>
I tried using response_type "id_
On Fri, Feb 16, 2018 at 7:25 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Nisha,
> Can you share details of "mycluster" topology ? also, can you turn up the
> logs to debug and share them along with the audit log that would help us to
> understand the problem better.
>
> Best,
> Sandeep
>
> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I have setup KNOX to connect with Azure AD using pac4j.
>>
>> However, after the authentication at Azure login page, it gets into an
>> infinite loop and does not give back the original REST call response.
>>
>> *Details:*
>>
>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>
>> 2. It redirects to *https://**login.microsoftonline.com
>> <http://login.microsoftonline.com/>* and asks for credentials.
>>
>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session
>> and state variables passed as below:
>>
>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>
>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>> <http://login.microsoftonline.com/>* like below:
>>
>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>
>> After this, step 1 and 2 alternate several times and finally lands up in
>> "ERR_TOO_MANY_REDIRECTS"!!!
>>
>> This is my knoxsso.xml:
>>
>>
>> 1. <topology>
>> 2. <gateway>
>> 3. <provider>
>> 4. <role>webappsec</role>
>> 5. <name>WebAppSec</name>
>> 6. <enabled>true</enabled>
>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>> 8. </provider>
>> 9. <provider>
>> 10. <role>federation</role>
>> 11. <name>pac4j</name>
>> 12. <enabled>true</enabled>
>> 13. <param>
>> 14. <name>pac4j.callbackUrl</name>
>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>> 16. </param>
>> 17. <param>
>> 18. <name>clientName</name>
>> 19. <value>OidcClient</value>
>> 20. </param>
>> 21. <param>
>> 22. <name>oidc.id</name>
>> 23. <value>385c2bc*****************2695eaa34</value>
>> 24. </param>
>> 25. <param>
>> 26. <name>oidc.secret</name>
>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>> 28. </param>
>> 29. <param>
>> 30. <name>oidc.discoveryUri</name>
>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>> 32. </param>
>> 33. </provider>
>> 34. <provider>
>> 35. <role>identity-assertion</role>
>> 36. <name>Default</name>
>> 37. <enabled>true</enabled>
>> 38. </provider>
>> 39. </gateway>
>> 40. <application>
>> 41. <name>knoxauth</name>
>> 42. </application>
>> 43. <service>
>> 44. <role>KNOXSSO</role>
>> 45. <param>
>> 46. <name>knoxsso.cookie.secure.only</name>
>> 47. <value>false</value>
>> 48. </param>
>> 49. <param>
>> 50. <name>knoxsso.token.ttl</name>
>> 51. <value>30000</value>
>> 52. </param>
>> 53. <param>
>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>> 56. </param>
>> 57. </service>
>> 58. </topology>
>>
>> I tried using response_type "id_token", enabling nonces, knoxsso.secure
>> to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>
>> gateway-audit.log when redirection error starts:
>>
>>
>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>
>> It clearly shows Response status as "302" and not "200". This leads to
>> redirection!
>>
>> What could I be missing here? Any pointers will be greatly appreciated.
>> Regards
>> Nisha
>>
>>
>
--
Nisha Menon
BTech (CS) Sahrdaya CET,
MTech (CS) IIIT Banglore.
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Yes, you are right.
I tested again the pac4j provider with client as "testBasicAuth". This
works perfectly and I noticed that pac4j provides a "hadoop-jwt" token at
the last step to the SSO, and then its able to get the results for the
original URL.
I also tested pac4j with basicAuth and casClient. And both works properly
(with hadoop-jwt passed by the pac4j provider)
Infact, pac4j with OidcClient (with auth0.com) also works properly.
So now the question is with OidcClient and Azure AD itself!
Not sure what else could wrong.
On Mon, Feb 19, 2018 at 9:46 PM, larry mccay <lm...@apache.org> wrote:
> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
> If it isn't being set, it could be that it isn't getting past the pac4j
> federation provider to the KnoxSSO service itself because there is a
> redirect from the pac4j provider or the Set-Cookie just isn't being
> accepted by the browser.
>
> The audit log does seem to be getting a redirect from pac4j.
>
> I haven't seen any examples of it working AAD - so we are in unchartered
> waters here.
>
> @Jerome - any insights?
>
> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello Larry,
>>
>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>> In SSO provider with pac4j, I can see cookies like:
>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>> pac4j.session.oidcNonceAttribute etc.
>>
>> IP address and hostnames are mapped, else the *basic auth* also would
>> have failed.
>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>
>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> It looks like you may be using ip addresses for your Knox URLs - to
>>> webhdfs.
>>> In order to rule out cookie related issue can you do a couple things:
>>>
>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>> browser
>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>> something that you can reference - I use www.local.com to map to
>>> localhost
>>>
>>> I think that ip address should work for this case actually but there are
>>> differences in browsers that might not let it.
>>> Also, if you had another service on another ip address, the browser
>>> would not present the cookie - so this is good to be aware of anyway.
>>>
>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> Hello Nisha,
>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>> the logs to debug and share them along with the audit log that would help
>>>> us to understand the problem better.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>
>>>>> However, after the authentication at Azure login page, it gets into an
>>>>> infinite loop and does not give back the original REST call response.
>>>>>
>>>>> *Details:*
>>>>>
>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>
>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>
>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>> session and state variables passed as below:
>>>>>
>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>
>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>
>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>
>>>>> After this, step 1 and 2 alternate several times and finally lands up
>>>>> in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>
>>>>> This is my knoxsso.xml:
>>>>>
>>>>>
>>>>> 1. <topology>
>>>>> 2. <gateway>
>>>>> 3. <provider>
>>>>> 4. <role>webappsec</role>
>>>>> 5. <name>WebAppSec</name>
>>>>> 6. <enabled>true</enabled>
>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>> 8. </provider>
>>>>> 9. <provider>
>>>>> 10. <role>federation</role>
>>>>> 11. <name>pac4j</name>
>>>>> 12. <enabled>true</enabled>
>>>>> 13. <param>
>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>> 16. </param>
>>>>> 17. <param>
>>>>> 18. <name>clientName</name>
>>>>> 19. <value>OidcClient</value>
>>>>> 20. </param>
>>>>> 21. <param>
>>>>> 22. <name>oidc.id</name>
>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>> 24. </param>
>>>>> 25. <param>
>>>>> 26. <name>oidc.secret</name>
>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>> 28. </param>
>>>>> 29. <param>
>>>>> 30. <name>oidc.discoveryUri</name>
>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>> 32. </param>
>>>>> 33. </provider>
>>>>> 34. <provider>
>>>>> 35. <role>identity-assertion</role>
>>>>> 36. <name>Default</name>
>>>>> 37. <enabled>true</enabled>
>>>>> 38. </provider>
>>>>> 39. </gateway>
>>>>> 40. <application>
>>>>> 41. <name>knoxauth</name>
>>>>> 42. </application>
>>>>> 43. <service>
>>>>> 44. <role>KNOXSSO</role>
>>>>> 45. <param>
>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>> 47. <value>false</value>
>>>>> 48. </param>
>>>>> 49. <param>
>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>> 51. <value>30000</value>
>>>>> 52. </param>
>>>>> 53. <param>
>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>> 56. </param>
>>>>> 57. </service>
>>>>> 58. </topology>
>>>>>
>>>>> I tried using response_type "id_token", enabling nonces,
>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>
>>>>> gateway-audit.log when redirection error starts:
>>>>>
>>>>>
>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>
>>>>> It clearly shows Response status as "302" and not "200". This leads to
>>>>> redirection!
>>>>>
>>>>> What could I be missing here? Any pointers will be greatly appreciated.
>>>>> Regards
>>>>> Nisha
>>>>>
>>>>>
>>>>
>>> Regards
>> Nisha
>>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Sure Larry, I will take a look at the links and will start working on an
extension right away.
On Tue, Feb 27, 2018 at 10:57 PM, larry mccay <lm...@apache.org> wrote:
> All -
>
> It is worth noting that I started KIP-11 [1] for cloud usecases for 1.1.0
> and have called out specific support for Azure AD.
> As noted in the KIP, we may need to consider a different provider for AAD
> integration using [2] as an example for the provider.
>
> @Nisha - it may be of interest to you to take a look at that example and
> to consider doing the provider implementation to unblock your usecase.
>
> We have a wiki [3] that describes what is involved with adding a new
> federation provider which is exactly what this would be.
>
> Working toward a solution through pac4j would also be important but the
> extensibility of Knox allows for flexibility when it is needed.
>
> thanks,
>
> --larry
>
> 1. https://cwiki.apache.org/confluence/display/KNOX/KIP-11+Cloud+Usecases
> 2. https://github.com/AzureAD/azure-activedirectory-library-for-java
> 3. https://cwiki.apache.org/confluence/display/KNOX/2015/
> 12/18/Adding+a+Federation+Provider+to+Apache+Knox
>
> On Tue, Feb 27, 2018 at 12:16 PM, Jérôme LELEU <le...@gmail.com> wrote:
>
>> Hi,
>>
>> I think you can't handle that currently.
>>
>> Generally, with pac4j v2, the solution is to define the client as the
>> default one and not to include the client_name parameter, which is not
>> possible for Knox currently.
>>
>> In pac4j v3, you'll be able to choose the PathParameterCallbackUrlRe
>> solver instead of the default QueryParameterCallbackUrlResolver component
>> to deal with that: having the client name in the URL and not as a
>> parameter. But it means it needs to be integrated with Knox.
>>
>> Thanks.
>> Best regards,
>> Jérôme
>>
>>
>> On Tue, Feb 27, 2018 at 9:51 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hi Jerome/Sandeep,
>>>
>>> Azure AD does support query parameters in the Reply URL for Azure app
>>> registration, whereas in v2 with microsoft app registration portal, it does
>>> not.
>>> However, even in the first case where it allows the parameters during
>>> configuration, it does not honor them by sending it back with the
>>> response.. hence the issue!
>>>
>>> I also came across a post ( https://groups.google.com/foru
>>> m/#!topic/pac4j-users/-GGzLrONSOI ) where the user tries to customize
>>> the call in such a way "/callback*/client_name=AzureAD/*" as opposed to
>>> " /callback*?client_name=AzureAD*". Can we try this quickly for KNOX?
>>> Is it possible to achieve this?
>>>
>>> Regards,
>>> Nisha
>>>
>>> On Mon, Feb 26, 2018 at 7:43 PM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> Hello Jérôme
>>>> You are right about the client_name and pac4JCallback parameters.
>>>> I do not think this is because of misconfiguration, the issue in this
>>>> case is that Azure AD does not support query params in the callback URLs.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>> On Mon, Feb 26, 2018 at 1:42 AM, Jérôme LELEU <le...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> The callback URL sent by the IdP does not have the client_name
>>>>> parameter, nor the pac4jCallback parameter (the parameter has changed
>>>>> between versions).
>>>>>
>>>>> So the callback is not properly detected and an authentication is
>>>>> tried over and over again.
>>>>>
>>>>> So I guess there is a misconfiguration on the IdP side at the callback
>>>>> URL level.
>>>>>
>>>>> Thanks.
>>>>> Best regards,
>>>>> Jérôme
>>>>>
>>>>>
>>>>> On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Jerome.
>>>>>>
>>>>>> PFA the logs after enabling the pac4j logs. Hope this helps.
>>>>>>
>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Logs look good in pac4j. Redirecting to the identity provider and
>>>>>>> then coming back to the gateway. Maybe the user identity is not properly
>>>>>>> created...
>>>>>>>
>>>>>>> Can you turn on DEBUG logs on org.pac4j and
>>>>>>> org.apache.knox.gateway.pac4j?
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Best regards,
>>>>>>> Jérôme
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <
>>>>>>> coheigea@apache.org> wrote:
>>>>>>>
>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>> following:
>>>>>>>>
>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>> pac4jRequestedUrl = null
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>> /api/v1/websso
>>>>>>>>
>>>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>>>> parameter and redirect to this instead?
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>> accepted by the browser.
>>>>>>>>>
>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>
>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>> unchartered waters here.
>>>>>>>>>
>>>>>>>>> @Jerome - any insights?
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Larry,
>>>>>>>>>>
>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>
>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>> would have failed.
>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>>>> to webhdfs.
>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>> things:
>>>>>>>>>>>
>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>> your browser
>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>
>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>> anyway.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>> response.
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>
>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>
>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>
>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Colm O hEigeartaigh
>>>>>>>>
>>>>>>>> Talend Community Coder
>>>>>>>> http://coders.talend.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>
--
Nisha Menon
BTech (CS) Sahrdaya CET,
MTech (CS) IIIT Banglore.
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
All -
It is worth noting that I started KIP-11 [1] for cloud usecases for 1.1.0
and have called out specific support for Azure AD.
As noted in the KIP, we may need to consider a different provider for AAD
integration using [2] as an example for the provider.
@Nisha - it may be of interest to you to take a look at that example and to
consider doing the provider implementation to unblock your usecase.
We have a wiki [3] that describes what is involved with adding a new
federation provider which is exactly what this would be.
Working toward a solution through pac4j would also be important but the
extensibility of Knox allows for flexibility when it is needed.
thanks,
--larry
1. https://cwiki.apache.org/confluence/display/KNOX/KIP-11+Cloud+Usecases
2. https://github.com/AzureAD/azure-activedirectory-library-for-java
3.
https://cwiki.apache.org/confluence/display/KNOX/2015/12/18/Adding+a+Federation+Provider+to+Apache+Knox
On Tue, Feb 27, 2018 at 12:16 PM, Jérôme LELEU <le...@gmail.com> wrote:
> Hi,
>
> I think you can't handle that currently.
>
> Generally, with pac4j v2, the solution is to define the client as the
> default one and not to include the client_name parameter, which is not
> possible for Knox currently.
>
> In pac4j v3, you'll be able to choose the PathParameterCallbackUrlResolver
> instead of the default QueryParameterCallbackUrlResolver component to
> deal with that: having the client name in the URL and not as a parameter.
> But it means it needs to be integrated with Knox.
>
> Thanks.
> Best regards,
> Jérôme
>
>
> On Tue, Feb 27, 2018 at 9:51 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hi Jerome/Sandeep,
>>
>> Azure AD does support query parameters in the Reply URL for Azure app
>> registration, whereas in v2 with microsoft app registration portal, it does
>> not.
>> However, even in the first case where it allows the parameters during
>> configuration, it does not honor them by sending it back with the
>> response.. hence the issue!
>>
>> I also came across a post ( https://groups.google.com/foru
>> m/#!topic/pac4j-users/-GGzLrONSOI ) where the user tries to customize
>> the call in such a way "/callback*/client_name=AzureAD/*" as opposed to
>> " /callback*?client_name=AzureAD*". Can we try this quickly for KNOX? Is
>> it possible to achieve this?
>>
>> Regards,
>> Nisha
>>
>> On Mon, Feb 26, 2018 at 7:43 PM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> Hello Jérôme
>>> You are right about the client_name and pac4JCallback parameters.
>>> I do not think this is because of misconfiguration, the issue in this
>>> case is that Azure AD does not support query params in the callback URLs.
>>>
>>> Best,
>>> Sandeep
>>>
>>> On Mon, Feb 26, 2018 at 1:42 AM, Jérôme LELEU <le...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> The callback URL sent by the IdP does not have the client_name
>>>> parameter, nor the pac4jCallback parameter (the parameter has changed
>>>> between versions).
>>>>
>>>> So the callback is not properly detected and an authentication is tried
>>>> over and over again.
>>>>
>>>> So I guess there is a misconfiguration on the IdP side at the callback
>>>> URL level.
>>>>
>>>> Thanks.
>>>> Best regards,
>>>> Jérôme
>>>>
>>>>
>>>> On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Jerome.
>>>>>
>>>>> PFA the logs after enabling the pac4j logs. Hope this helps.
>>>>>
>>>>> Regards
>>>>> Nisha
>>>>>
>>>>> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Logs look good in pac4j. Redirecting to the identity provider and
>>>>>> then coming back to the gateway. Maybe the user identity is not properly
>>>>>> created...
>>>>>>
>>>>>> Can you turn on DEBUG logs on org.pac4j and
>>>>>> org.apache.knox.gateway.pac4j?
>>>>>>
>>>>>> Thanks.
>>>>>> Best regards,
>>>>>> Jérôme
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>> following:
>>>>>>>
>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>> pac4jRequestedUrl = null
>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>>>> redirectUrl: https://localhost:8443/gateway
>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>> /api/v1/websso
>>>>>>>
>>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>>> parameter and redirect to this instead?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>> accepted by the browser.
>>>>>>>>
>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>
>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>> unchartered waters here.
>>>>>>>>
>>>>>>>> @Jerome - any insights?
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Larry,
>>>>>>>>>
>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>
>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>> would have failed.
>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>>> to webhdfs.
>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>> things:
>>>>>>>>>>
>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>> your browser
>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>
>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>> anyway.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Sandeep
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>
>>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>>> response.
>>>>>>>>>>>>
>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>
>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>
>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>
>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>
>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>
>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>
>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>
>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>
>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>
>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>
>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>> appreciated.
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Nisha
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>> Nisha
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Jérôme LELEU <le...@gmail.com>.
Hi,
I think you can't handle that currently.
Generally, with pac4j v2, the solution is to define the client as the
default one and not to include the client_name parameter, which is not
possible for Knox currently.
In pac4j v3, you'll be able to choose the PathParameterCallbackUrlResolver
instead of the default QueryParameterCallbackUrlResolver component to deal
with that: having the client name in the URL and not as a parameter. But it
means it needs to be integrated with Knox.
Thanks.
Best regards,
Jérôme
On Tue, Feb 27, 2018 at 9:51 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Hi Jerome/Sandeep,
>
> Azure AD does support query parameters in the Reply URL for Azure app
> registration, whereas in v2 with microsoft app registration portal, it does
> not.
> However, even in the first case where it allows the parameters during
> configuration, it does not honor them by sending it back with the
> response.. hence the issue!
>
> I also came across a post ( https://groups.google.com/
> forum/#!topic/pac4j-users/-GGzLrONSOI ) where the user tries to customize
> the call in such a way "/callback*/client_name=AzureAD/*" as opposed to "
> /callback*?client_name=AzureAD*". Can we try this quickly for KNOX? Is it
> possible to achieve this?
>
> Regards,
> Nisha
>
> On Mon, Feb 26, 2018 at 7:43 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Jérôme
>> You are right about the client_name and pac4JCallback parameters.
>> I do not think this is because of misconfiguration, the issue in this
>> case is that Azure AD does not support query params in the callback URLs.
>>
>> Best,
>> Sandeep
>>
>> On Mon, Feb 26, 2018 at 1:42 AM, Jérôme LELEU <le...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> The callback URL sent by the IdP does not have the client_name
>>> parameter, nor the pac4jCallback parameter (the parameter has changed
>>> between versions).
>>>
>>> So the callback is not properly detected and an authentication is tried
>>> over and over again.
>>>
>>> So I guess there is a misconfiguration on the IdP side at the callback
>>> URL level.
>>>
>>> Thanks.
>>> Best regards,
>>> Jérôme
>>>
>>>
>>> On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hi Jerome.
>>>>
>>>> PFA the logs after enabling the pac4j logs. Hope this helps.
>>>>
>>>> Regards
>>>> Nisha
>>>>
>>>> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Logs look good in pac4j. Redirecting to the identity provider and then
>>>>> coming back to the gateway. Maybe the user identity is not properly
>>>>> created...
>>>>>
>>>>> Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pa
>>>>> c4j?
>>>>>
>>>>> Thanks.
>>>>> Best regards,
>>>>> Jérôme
>>>>>
>>>>>
>>>>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <
>>>>> coheigea@apache.org> wrote:
>>>>>
>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>> following:
>>>>>>
>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>> pac4jRequestedUrl = null
>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>>> redirectUrl: https://localhost:8443/gateway
>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>> /api/v1/websso
>>>>>>
>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>> parameter and redirect to this instead?
>>>>>>
>>>>>> Colm.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>> accepted by the browser.
>>>>>>>
>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>
>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>> unchartered waters here.
>>>>>>>
>>>>>>> @Jerome - any insights?
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello Larry,
>>>>>>>>
>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>
>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>> would have failed.
>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>> to webhdfs.
>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>> things:
>>>>>>>>>
>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>> your browser
>>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>>> to localhost
>>>>>>>>>
>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>> anyway.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Nisha,
>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Sandeep
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>
>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>> response.
>>>>>>>>>>>
>>>>>>>>>>> *Details:*
>>>>>>>>>>>
>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>
>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>
>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>>
>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>
>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>
>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>
>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>
>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 1. <topology>
>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>> 3. <provider>
>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>> 8. </provider>
>>>>>>>>>>> 9. <provider>
>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>> 13. <param>
>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>> 16. </param>
>>>>>>>>>>> 17. <param>
>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>> 20. </param>
>>>>>>>>>>> 21. <param>
>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>> 24. </param>
>>>>>>>>>>> 25. <param>
>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>> 28. </param>
>>>>>>>>>>> 29. <param>
>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>> 32. </param>
>>>>>>>>>>> 33. </provider>
>>>>>>>>>>> 34. <provider>
>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>> 38. </provider>
>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>> 40. <application>
>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>> 42. </application>
>>>>>>>>>>> 43. <service>
>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>> 45. <param>
>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>> 48. </param>
>>>>>>>>>>> 49. <param>
>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>> 52. </param>
>>>>>>>>>>> 53. <param>
>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>> 56. </param>
>>>>>>>>>>> 57. </service>
>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>
>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>
>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>
>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>
>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>> appreciated.
>>>>>>>>>>> Regards
>>>>>>>>>>> Nisha
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>> Nisha
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Colm O hEigeartaigh
>>>>>>
>>>>>> Talend Community Coder
>>>>>> http://coders.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hi Jerome/Sandeep,
Azure AD does support query parameters in the Reply URL for Azure app
registration, whereas in v2 with microsoft app registration portal, it does
not.
However, even in the first case where it allows the parameters during
configuration, it does not honor them by sending it back with the
response.. hence the issue!
I also came across a post (
https://groups.google.com/forum/#!topic/pac4j-users/-GGzLrONSOI ) where the
user tries to customize the call in such a way "/callback
*/client_name=AzureAD/*" as opposed to " /callback*?client_name=AzureAD*".
Can we try this quickly for KNOX? Is it possible to achieve this?
Regards,
Nisha
On Mon, Feb 26, 2018 at 7:43 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Jérôme
> You are right about the client_name and pac4JCallback parameters.
> I do not think this is because of misconfiguration, the issue in this case
> is that Azure AD does not support query params in the callback URLs.
>
> Best,
> Sandeep
>
> On Mon, Feb 26, 2018 at 1:42 AM, Jérôme LELEU <le...@gmail.com> wrote:
>
>> Hi,
>>
>> The callback URL sent by the IdP does not have the client_name parameter,
>> nor the pac4jCallback parameter (the parameter has changed between
>> versions).
>>
>> So the callback is not properly detected and an authentication is tried
>> over and over again.
>>
>> So I guess there is a misconfiguration on the IdP side at the callback
>> URL level.
>>
>> Thanks.
>> Best regards,
>> Jérôme
>>
>>
>> On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hi Jerome.
>>>
>>> PFA the logs after enabling the pac4j logs. Hope this helps.
>>>
>>> Regards
>>> Nisha
>>>
>>> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Logs look good in pac4j. Redirecting to the identity provider and then
>>>> coming back to the gateway. Maybe the user identity is not properly
>>>> created...
>>>>
>>>> Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pa
>>>> c4j?
>>>>
>>>> Thanks.
>>>> Best regards,
>>>> Jérôme
>>>>
>>>>
>>>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>> following:
>>>>>
>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>>> = null
>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>> redirectUrl: https://localhost:8443/gateway
>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>> /api/v1/websso
>>>>>
>>>>> It is getting an access token correctly and trying to redirect back to
>>>>> the original URL. However from the logs above it seems to never hit the
>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>> parameter and redirect to this instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>> accepted by the browser.
>>>>>>
>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>
>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>> unchartered waters here.
>>>>>>
>>>>>> @Jerome - any insights?
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>
>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>> would have failed.
>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>>> webhdfs.
>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>> things:
>>>>>>>>
>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>>> browser
>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>> to localhost
>>>>>>>>
>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Nisha,
>>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>>> help us to understand the problem better.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Sandeep
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>
>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>> response.
>>>>>>>>>>
>>>>>>>>>> *Details:*
>>>>>>>>>>
>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>
>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>
>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>
>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>
>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>
>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>
>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>
>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. <topology>
>>>>>>>>>> 2. <gateway>
>>>>>>>>>> 3. <provider>
>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>> 8. </provider>
>>>>>>>>>> 9. <provider>
>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>> 13. <param>
>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>> 16. </param>
>>>>>>>>>> 17. <param>
>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>> 20. </param>
>>>>>>>>>> 21. <param>
>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>> 24. </param>
>>>>>>>>>> 25. <param>
>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>> 28. </param>
>>>>>>>>>> 29. <param>
>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>> 32. </param>
>>>>>>>>>> 33. </provider>
>>>>>>>>>> 34. <provider>
>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>> 38. </provider>
>>>>>>>>>> 39. </gateway>
>>>>>>>>>> 40. <application>
>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>> 42. </application>
>>>>>>>>>> 43. <service>
>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>> 45. <param>
>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>> 48. </param>
>>>>>>>>>> 49. <param>
>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>> 52. </param>
>>>>>>>>>> 53. <param>
>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>> 56. </param>
>>>>>>>>>> 57. </service>
>>>>>>>>>> 58. </topology>
>>>>>>>>>>
>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>
>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>
>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>> leads to redirection!
>>>>>>>>>>
>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>> appreciated.
>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Jérôme
You are right about the client_name and pac4JCallback parameters.
I do not think this is because of misconfiguration, the issue in this case
is that Azure AD does not support query params in the callback URLs.
Best,
Sandeep
On Mon, Feb 26, 2018 at 1:42 AM, Jérôme LELEU <le...@gmail.com> wrote:
> Hi,
>
> The callback URL sent by the IdP does not have the client_name parameter,
> nor the pac4jCallback parameter (the parameter has changed between
> versions).
>
> So the callback is not properly detected and an authentication is tried
> over and over again.
>
> So I guess there is a misconfiguration on the IdP side at the callback URL
> level.
>
> Thanks.
> Best regards,
> Jérôme
>
>
> On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hi Jerome.
>>
>> PFA the logs after enabling the pac4j logs. Hope this helps.
>>
>> Regards
>> Nisha
>>
>> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Logs look good in pac4j. Redirecting to the identity provider and then
>>> coming back to the gateway. Maybe the user identity is not properly
>>> created...
>>>
>>> Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pa
>>> c4j?
>>>
>>> Thanks.
>>> Best regards,
>>> Jérôme
>>>
>>>
>>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> I can reproduce the issue with Google. From the logs I see the
>>>> following:
>>>>
>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>> = null
>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>> redirectUrl: https://localhost:8443/gateway
>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>> /api/v1/websso
>>>>
>>>> It is getting an access token correctly and trying to redirect back to
>>>> the original URL. However from the logs above it seems to never hit the
>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>> parameter and redirect to this instead?
>>>>
>>>> Colm.
>>>>
>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>> accepted by the browser.
>>>>>
>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>
>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>> unchartered waters here.
>>>>>
>>>>> @Jerome - any insights?
>>>>>
>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Larry,
>>>>>>
>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>
>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>> would have failed.
>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>> webhdfs.
>>>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>>>
>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>> browser
>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>> or something that you can reference - I use www.local.com to map to
>>>>>>> localhost
>>>>>>>
>>>>>>> I think that ip address should work for this case actually but there
>>>>>>> are differences in browsers that might not let it.
>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <moresandeep@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hello Nisha,
>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>> help us to understand the problem better.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Sandeep
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>
>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>> response.
>>>>>>>>>
>>>>>>>>> *Details:*
>>>>>>>>>
>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>
>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>
>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>> session and state variables passed as below:
>>>>>>>>>
>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>
>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>
>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>
>>>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>
>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. <topology>
>>>>>>>>> 2. <gateway>
>>>>>>>>> 3. <provider>
>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>> 8. </provider>
>>>>>>>>> 9. <provider>
>>>>>>>>> 10. <role>federation</role>
>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>> 13. <param>
>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>> 16. </param>
>>>>>>>>> 17. <param>
>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>> 20. </param>
>>>>>>>>> 21. <param>
>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>> 24. </param>
>>>>>>>>> 25. <param>
>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>> 28. </param>
>>>>>>>>> 29. <param>
>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>> 32. </param>
>>>>>>>>> 33. </provider>
>>>>>>>>> 34. <provider>
>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>> 36. <name>Default</name>
>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>> 38. </provider>
>>>>>>>>> 39. </gateway>
>>>>>>>>> 40. <application>
>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>> 42. </application>
>>>>>>>>> 43. <service>
>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>> 45. <param>
>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>> 47. <value>false</value>
>>>>>>>>> 48. </param>
>>>>>>>>> 49. <param>
>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>> 51. <value>30000</value>
>>>>>>>>> 52. </param>
>>>>>>>>> 53. <param>
>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>> 56. </param>
>>>>>>>>> 57. </service>
>>>>>>>>> 58. </topology>
>>>>>>>>>
>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>
>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>
>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>> leads to redirection!
>>>>>>>>>
>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>> appreciated.
>>>>>>>>> Regards
>>>>>>>>> Nisha
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Jérôme LELEU <le...@gmail.com>.
Hi,
The callback URL sent by the IdP does not have the client_name parameter,
nor the pac4jCallback parameter (the parameter has changed between
versions).
So the callback is not properly detected and an authentication is tried
over and over again.
So I guess there is a misconfiguration on the IdP side at the callback URL
level.
Thanks.
Best regards,
Jérôme
On Wed, Feb 21, 2018 at 4:45 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Hi Jerome.
>
> PFA the logs after enabling the pac4j logs. Hope this helps.
>
> Regards
> Nisha
>
> On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com> wrote:
>
>> Hi,
>>
>> Logs look good in pac4j. Redirecting to the identity provider and then
>> coming back to the gateway. Maybe the user identity is not properly
>> created...
>>
>> Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pa
>> c4j?
>>
>> Thanks.
>> Best regards,
>> Jérôme
>>
>>
>> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> I can reproduce the issue with Google. From the logs I see the following:
>>>
>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
>>> https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
>>> null
>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>> redirectUrl: https://localhost:8443/gateway
>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
>>> - Received request: GET /api/v1/websso
>>>
>>> It is getting an access token correctly and trying to redirect back to
>>> the original URL. However from the logs above it seems to never hit the
>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>> parameter and redirect to this instead?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>> If it isn't being set, it could be that it isn't getting past the pac4j
>>>> federation provider to the KnoxSSO service itself because there is a
>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>> accepted by the browser.
>>>>
>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>
>>>> I haven't seen any examples of it working AAD - so we are in
>>>> unchartered waters here.
>>>>
>>>> @Jerome - any insights?
>>>>
>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Larry,
>>>>>
>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>
>>>>> IP address and hostnames are mapped, else the *basic auth* also would
>>>>> have failed.
>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>
>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>> webhdfs.
>>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>>
>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>> browser
>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>>>> something that you can reference - I use www.local.com to map to
>>>>>> localhost
>>>>>>
>>>>>> I think that ip address should work for this case actually but there
>>>>>> are differences in browsers that might not let it.
>>>>>> Also, if you had another service on another ip address, the browser
>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Nisha,
>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>> help us to understand the problem better.
>>>>>>>
>>>>>>> Best,
>>>>>>> Sandeep
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>
>>>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>>>
>>>>>>>> *Details:*
>>>>>>>>
>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>
>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>
>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>> session and state variables passed as below:
>>>>>>>>
>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>
>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>
>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>
>>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>
>>>>>>>> This is my knoxsso.xml:
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. <topology>
>>>>>>>> 2. <gateway>
>>>>>>>> 3. <provider>
>>>>>>>> 4. <role>webappsec</role>
>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>> 8. </provider>
>>>>>>>> 9. <provider>
>>>>>>>> 10. <role>federation</role>
>>>>>>>> 11. <name>pac4j</name>
>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>> 13. <param>
>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>> 16. </param>
>>>>>>>> 17. <param>
>>>>>>>> 18. <name>clientName</name>
>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>> 20. </param>
>>>>>>>> 21. <param>
>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>> 24. </param>
>>>>>>>> 25. <param>
>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>> 28. </param>
>>>>>>>> 29. <param>
>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>> 32. </param>
>>>>>>>> 33. </provider>
>>>>>>>> 34. <provider>
>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>> 36. <name>Default</name>
>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>> 38. </provider>
>>>>>>>> 39. </gateway>
>>>>>>>> 40. <application>
>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>> 42. </application>
>>>>>>>> 43. <service>
>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>> 45. <param>
>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>> 47. <value>false</value>
>>>>>>>> 48. </param>
>>>>>>>> 49. <param>
>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>> 51. <value>30000</value>
>>>>>>>> 52. </param>
>>>>>>>> 53. <param>
>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>> 56. </param>
>>>>>>>> 57. </service>
>>>>>>>> 58. </topology>
>>>>>>>>
>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>
>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>
>>>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>>>> to redirection!
>>>>>>>>
>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>> appreciated.
>>>>>>>> Regards
>>>>>>>> Nisha
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> Regards
>>>>> Nisha
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hi Jerome.
PFA the logs after enabling the pac4j logs. Hope this helps.
Regards
Nisha
On Tue, Feb 20, 2018 at 7:50 PM, Jérôme LELEU <le...@gmail.com> wrote:
> Hi,
>
> Logs look good in pac4j. Redirecting to the identity provider and then
> coming back to the gateway. Maybe the user identity is not properly
> created...
>
> Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pac4j?
>
> Thanks.
> Best regards,
> Jérôme
>
>
> On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
>> I can reproduce the issue with Google. From the logs I see the following:
>>
>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
>> https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?
>> originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/
>> webhdfs/v1/data/LICENSE.txt?op=OPEN
>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
>> null
>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>> redirectUrl: https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?
>> originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/
>> webhdfs/v1/data/LICENSE.txt?op=OPEN
>> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
>> - Received request: GET /api/v1/websso
>>
>> It is getting an access token correctly and trying to redirect back to
>> the original URL. However from the logs above it seems to never hit the
>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>> parameter and redirect to this instead?
>>
>> Colm.
>>
>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>> If it isn't being set, it could be that it isn't getting past the pac4j
>>> federation provider to the KnoxSSO service itself because there is a
>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>> accepted by the browser.
>>>
>>> The audit log does seem to be getting a redirect from pac4j.
>>>
>>> I haven't seen any examples of it working AAD - so we are in unchartered
>>> waters here.
>>>
>>> @Jerome - any insights?
>>>
>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hello Larry,
>>>>
>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>> In SSO provider with pac4j, I can see cookies like:
>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>> pac4j.session.oidcNonceAttribute etc.
>>>>
>>>> IP address and hostnames are mapped, else the *basic auth* also would
>>>> have failed.
>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>
>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>> wrote:
>>>>
>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>> webhdfs.
>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>
>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>> browser
>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>>> something that you can reference - I use www.local.com to map to
>>>>> localhost
>>>>>
>>>>> I think that ip address should work for this case actually but there
>>>>> are differences in browsers that might not let it.
>>>>> Also, if you had another service on another ip address, the browser
>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>
>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Nisha,
>>>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>>>> the logs to debug and share them along with the audit log that would help
>>>>>> us to understand the problem better.
>>>>>>
>>>>>> Best,
>>>>>> Sandeep
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>
>>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>>
>>>>>>> *Details:*
>>>>>>>
>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>
>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>
>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>> session and state variables passed as below:
>>>>>>>
>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>
>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>
>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>
>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>
>>>>>>> This is my knoxsso.xml:
>>>>>>>
>>>>>>>
>>>>>>> 1. <topology>
>>>>>>> 2. <gateway>
>>>>>>> 3. <provider>
>>>>>>> 4. <role>webappsec</role>
>>>>>>> 5. <name>WebAppSec</name>
>>>>>>> 6. <enabled>true</enabled>
>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>> 8. </provider>
>>>>>>> 9. <provider>
>>>>>>> 10. <role>federation</role>
>>>>>>> 11. <name>pac4j</name>
>>>>>>> 12. <enabled>true</enabled>
>>>>>>> 13. <param>
>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>> 16. </param>
>>>>>>> 17. <param>
>>>>>>> 18. <name>clientName</name>
>>>>>>> 19. <value>OidcClient</value>
>>>>>>> 20. </param>
>>>>>>> 21. <param>
>>>>>>> 22. <name>oidc.id</name>
>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>> 24. </param>
>>>>>>> 25. <param>
>>>>>>> 26. <name>oidc.secret</name>
>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>> 28. </param>
>>>>>>> 29. <param>
>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>> 32. </param>
>>>>>>> 33. </provider>
>>>>>>> 34. <provider>
>>>>>>> 35. <role>identity-assertion</role>
>>>>>>> 36. <name>Default</name>
>>>>>>> 37. <enabled>true</enabled>
>>>>>>> 38. </provider>
>>>>>>> 39. </gateway>
>>>>>>> 40. <application>
>>>>>>> 41. <name>knoxauth</name>
>>>>>>> 42. </application>
>>>>>>> 43. <service>
>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>> 45. <param>
>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>> 47. <value>false</value>
>>>>>>> 48. </param>
>>>>>>> 49. <param>
>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>> 51. <value>30000</value>
>>>>>>> 52. </param>
>>>>>>> 53. <param>
>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>> 56. </param>
>>>>>>> 57. </service>
>>>>>>> 58. </topology>
>>>>>>>
>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>
>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>
>>>>>>>
>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>
>>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>>> to redirection!
>>>>>>>
>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>> appreciated.
>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>>
>>>>>>
>>>>> Regards
>>>> Nisha
>>>>
>>>
>>>
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Jérôme LELEU <le...@gmail.com>.
Hi,
Logs look good in pac4j. Redirecting to the identity provider and then
coming back to the gateway. Maybe the user identity is not properly
created...
Can you turn on DEBUG logs on org.pac4j and org.apache.knox.gateway.pac4j?
Thanks.
Best regards,
Jérôme
On Mon, Feb 19, 2018 at 6:49 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> I can reproduce the issue with Google. From the logs I see the following:
>
> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
> https://localhost:8443/gateway/knoxssopac4j/api/v1/
> websso?originalUrl=https://localhost:8443/gateway/
> sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
> null
> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
> redirectUrl: https://localhost:8443/gateway/knoxssopac4j/api/v1/
> websso?originalUrl=https://localhost:8443/gateway/
> sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
> - Received request: GET /api/v1/websso
>
> It is getting an access token correctly and trying to redirect back to the
> original URL. However from the logs above it seems to never hit the
> original URL but instead hits the "redirectUrl". Is some parsing supposed
> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
> parameter and redirect to this instead?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>
>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>> If it isn't being set, it could be that it isn't getting past the pac4j
>> federation provider to the KnoxSSO service itself because there is a
>> redirect from the pac4j provider or the Set-Cookie just isn't being
>> accepted by the browser.
>>
>> The audit log does seem to be getting a redirect from pac4j.
>>
>> I haven't seen any examples of it working AAD - so we are in unchartered
>> waters here.
>>
>> @Jerome - any insights?
>>
>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello Larry,
>>>
>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>> In SSO provider with pac4j, I can see cookies like:
>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>> pac4j.session.oidcNonceAttribute etc.
>>>
>>> IP address and hostnames are mapped, else the *basic auth* also would
>>> have failed.
>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>
>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>> webhdfs.
>>>> In order to rule out cookie related issue can you do a couple things:
>>>>
>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>> browser
>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>> something that you can reference - I use www.local.com to map to
>>>> localhost
>>>>
>>>> I think that ip address should work for this case actually but there
>>>> are differences in browsers that might not let it.
>>>> Also, if you had another service on another ip address, the browser
>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>
>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Nisha,
>>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>>> the logs to debug and share them along with the audit log that would help
>>>>> us to understand the problem better.
>>>>>
>>>>> Best,
>>>>> Sandeep
>>>>>
>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>
>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>
>>>>>> *Details:*
>>>>>>
>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>
>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>
>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>> session and state variables passed as below:
>>>>>>
>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>
>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>
>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>
>>>>>> After this, step 1 and 2 alternate several times and finally lands up
>>>>>> in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>
>>>>>> This is my knoxsso.xml:
>>>>>>
>>>>>>
>>>>>> 1. <topology>
>>>>>> 2. <gateway>
>>>>>> 3. <provider>
>>>>>> 4. <role>webappsec</role>
>>>>>> 5. <name>WebAppSec</name>
>>>>>> 6. <enabled>true</enabled>
>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>> 8. </provider>
>>>>>> 9. <provider>
>>>>>> 10. <role>federation</role>
>>>>>> 11. <name>pac4j</name>
>>>>>> 12. <enabled>true</enabled>
>>>>>> 13. <param>
>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>> 16. </param>
>>>>>> 17. <param>
>>>>>> 18. <name>clientName</name>
>>>>>> 19. <value>OidcClient</value>
>>>>>> 20. </param>
>>>>>> 21. <param>
>>>>>> 22. <name>oidc.id</name>
>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>> 24. </param>
>>>>>> 25. <param>
>>>>>> 26. <name>oidc.secret</name>
>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>> 28. </param>
>>>>>> 29. <param>
>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>> 32. </param>
>>>>>> 33. </provider>
>>>>>> 34. <provider>
>>>>>> 35. <role>identity-assertion</role>
>>>>>> 36. <name>Default</name>
>>>>>> 37. <enabled>true</enabled>
>>>>>> 38. </provider>
>>>>>> 39. </gateway>
>>>>>> 40. <application>
>>>>>> 41. <name>knoxauth</name>
>>>>>> 42. </application>
>>>>>> 43. <service>
>>>>>> 44. <role>KNOXSSO</role>
>>>>>> 45. <param>
>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>> 47. <value>false</value>
>>>>>> 48. </param>
>>>>>> 49. <param>
>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>> 51. <value>30000</value>
>>>>>> 52. </param>
>>>>>> 53. <param>
>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>> 56. </param>
>>>>>> 57. </service>
>>>>>> 58. </topology>
>>>>>>
>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>
>>>>>> gateway-audit.log when redirection error starts:
>>>>>>
>>>>>>
>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>
>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>> to redirection!
>>>>>>
>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>> appreciated.
>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>>
>>>>>
>>>> Regards
>>> Nisha
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
Ahhh - didn't realize that you were on 0.12.0.
Yes, we upgraded in 0.14.0/1.0.0 - I think that there were other dependency
changes as a result.
I am trying with 1.0.0 now.
On Tue, Feb 20, 2018 at 9:11 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Also, can I try replacing pac4j-oidc jar in KNOX deployment? Would this
> bring up the newly available clients?
>
> These are the existing jars for KNOX 0.12.0 (that came with HDP 2.7):
> /usr/hdp/2.6.1.0-129/knox/dep/oauth2-oidc-sdk-5.1.jar
> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oidc-1.8.9.jar
>
> Regards
> Nisha
>
>
> On Tue, Feb 20, 2018 at 7:13 PM, larry mccay <lm...@apache.org> wrote:
>
>> Oh, bummer.
>>
>> @Colm - can you share how you changed the KnoxSessionStore with the
>> J2ESessionStore?
>>
>>
>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello Larry,
>>>
>>> Well, this is what I started with in my implementation. I tested again.
>>> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
>>> be recongnised in pac4j client.
>>> Thats when I used "OidcClient" itself".
>>> Error:
>>>
>>> Caused by: org.pac4j.core.exception.TechnicalException: No client found
>>> for name: AzureAdClient
>>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>>> at org.pac4j.core.client.DefaultClientFinder.find(DefaultClient
>>> Finder.java:56)
>>>
>>> Regards
>>> Nisha
>>>
>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> So, it seems that OidcClient and Auth0 is working but AAD and Google
>>>> are not.
>>>> While I've never tested AAD, I have tested Google so there may be a
>>>> regression there.
>>>>
>>>> I notice that there are specific clients for both AzureAd and Google
>>>> now - see: http://www.pac4j.org/docs/clients/openid-connect.html#2
>>>> -clients
>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>
>>>> They also have corresponding profile creators.
>>>>
>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>> AzureAdOidcClient ?
>>>>
>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> Trying with "www.local.com" makes no difference for me. The problem
>>>>> is that it is not retrieving the stored user profile correctly after it
>>>>> redirects back to the Pac4jDispatcherFilter:
>>>>>
>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession:
>>>>> true
>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles
>>>>> = null
>>>>>
>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>
>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>
>>>>> and it worked! Perhaps we should change from storing the profile in a
>>>>> cookie to an attribute on the session instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>> originally requested resource.
>>>>>>
>>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>>> that authentication/federation of the enduser has been done, that the
>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>> mapped it to whatever the effective user should be.
>>>>>>
>>>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>>>> from the normalized Java Subject, create the JWT representation of the
>>>>>> authentication event and set-cookie.
>>>>>> It then redirects the user agent to the originally requested resource
>>>>>> where the browser should present the cookie.
>>>>>>
>>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>>> is referenced in:
>>>>>>>
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/js/knoxauth.js
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/redirecting.jsp
>>>>>>>
>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Colm -
>>>>>>>>
>>>>>>>> Thanks for trying to reproduce.
>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>
>>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>
>>>>>>>> Can you switch to a full domain name?
>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>> file.
>>>>>>>>
>>>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>>>> name.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> --larry
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>> following:
>>>>>>>>>
>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>> /api/v1/websso
>>>>>>>>>
>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>> accepted by the browser.
>>>>>>>>>>
>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>
>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>> unchartered waters here.
>>>>>>>>>>
>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>
>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>
>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>>> would have failed.
>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lmccay@apache.org
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs
>>>>>>>>>>>> - to webhdfs.
>>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>>> things:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>>> your browser
>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>
>>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>> anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>> Nisha
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Also, can I try replacing pac4j-oidc jar in KNOX deployment? Would this
bring up the newly available clients?
These are the existing jars for KNOX 0.12.0 (that came with HDP 2.7):
/usr/hdp/2.6.1.0-129/knox/dep/oauth2-oidc-sdk-5.1.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-oidc-1.8.9.jar
Regards
Nisha
On Tue, Feb 20, 2018 at 7:13 PM, larry mccay <lm...@apache.org> wrote:
> Oh, bummer.
>
> @Colm - can you share how you changed the KnoxSessionStore with the
> J2ESessionStore?
>
>
> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello Larry,
>>
>> Well, this is what I started with in my implementation. I tested again.
>> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
>> be recongnised in pac4j client.
>> Thats when I used "OidcClient" itself".
>> Error:
>>
>> Caused by: org.pac4j.core.exception.TechnicalException: No client found
>> for name: AzureAdClient
>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>> at org.pac4j.core.client.DefaultClientFinder.find(DefaultClient
>> Finder.java:56)
>>
>> Regards
>> Nisha
>>
>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> So, it seems that OidcClient and Auth0 is working but AAD and Google are
>>> not.
>>> While I've never tested AAD, I have tested Google so there may be a
>>> regression there.
>>>
>>> I notice that there are specific clients for both AzureAd and Google now
>>> - see: http://www.pac4j.org/docs/clients/openid-connect.html#2-clients
>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>
>>> They also have corresponding profile creators.
>>>
>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>> AzureAdOidcClient ?
>>>
>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> Trying with "www.local.com" makes no difference for me. The problem is
>>>> that it is not retrieving the stored user profile correctly after it
>>>> redirects back to the Pac4jDispatcherFilter:
>>>>
>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles
>>>> = null
>>>>
>>>> However, I replaced the KnoxSessionStore with the following:
>>>>
>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>
>>>> and it worked! Perhaps we should change from storing the profile in a
>>>> cookie to an attribute on the session instead?
>>>>
>>>> Colm.
>>>>
>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>>>>> requested resource.
>>>>>
>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>> that authentication/federation of the enduser has been done, that the
>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>> mapped it to whatever the effective user should be.
>>>>>
>>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>>> from the normalized Java Subject, create the JWT representation of the
>>>>> authentication event and set-cookie.
>>>>> It then redirects the user agent to the originally requested resource
>>>>> where the browser should present the cookie.
>>>>>
>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>> coheigea@apache.org> wrote:
>>>>>
>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>> is referenced in:
>>>>>>
>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>> h/app/js/knoxauth.js
>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>> h/app/redirecting.jsp
>>>>>>
>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>
>>>>>> Colm.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Colm -
>>>>>>>
>>>>>>> Thanks for trying to reproduce.
>>>>>>> Starting to get concerned about a regression....
>>>>>>>
>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>
>>>>>>> Can you switch to a full domain name?
>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>>>>
>>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>>> name.
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> --larry
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>> coheigea@apache.org> wrote:
>>>>>>>
>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>> following:
>>>>>>>>
>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>> pac4jRequestedUrl = null
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>> /api/v1/websso
>>>>>>>>
>>>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>>>> parameter and redirect to this instead?
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>> accepted by the browser.
>>>>>>>>>
>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>
>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>> unchartered waters here.
>>>>>>>>>
>>>>>>>>> @Jerome - any insights?
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Larry,
>>>>>>>>>>
>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>
>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>> would have failed.
>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>>>> to webhdfs.
>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>> things:
>>>>>>>>>>>
>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>> your browser
>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>
>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>> anyway.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>> response.
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>
>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>
>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>
>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Colm O hEigeartaigh
>>>>>>>>
>>>>>>>> Talend Community Coder
>>>>>>>> http://coders.talend.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Colm O hEigeartaigh
>>>>>>
>>>>>> Talend Community Coder
>>>>>> http://coders.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
Thanks Larry !
I do like the idea of umbrella bug especially given that there is no design
change here.
The tracking JIRA is https://issues.apache.org/jira/browse/KNOX-1189
Best,
Sandeep
On Thu, Feb 22, 2018 at 10:41 AM, larry mccay <lm...@apache.org> wrote:
> All -
>
> This is an unfortunate situation but a great triage effort!
>
> I think this could be captured in a KIP as Sandeep suggests - especially
> if it requires the communication of some design or alternative approaches.
> It may also be sufficient to track multiple issues with an umbrella JIRA
> if this seems like more of a list of bugs and improvements.
>
> I have been a bit delayed in getting out a planning thread for 1.1.0 but I
> do think that these issues need to addressed there.
> Especially the regression for Google.
>
> Azure AD is a new pac4j feature at least which we hadn't ever tried before.
> SAML based integration with aad may still actually work.
>
> thanks!
>
> --larry
>
>
> On Thu, Feb 22, 2018 at 9:15 AM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Colm,
>>
>> Thanks for summarizing, you are right with a) and b)
>> about c) though I think it is doing the right thing, the issue I saw in
>> my debugging was that the callback filter for Pac4J was never hit (absence
>> of callback param), as a result the request from IDP would go to security
>> filter and then get redirected to IDP, IDP had already authenticated so it
>> redirects back to Knox, hence the loop.
>>
>> I would like to add few things to your list that I found out:
>>
>> d) Support for "oidc.type" [2], this is the property that chooses Azure
>> or Google client, although not sure how much it will help.
>> e) Config variables "defaultUrl", "multiProfile" and "renewSession" are
>> not getting picked up.
>>
>> I think we need a separate KIP to track all these issues. On the face of
>> it, a) seems easy to fix and we can get folks unblocked asap.
>>
>> I really hate that Azure does not honor the callback parameters, it is
>> frustrating trying to debug the problem without a proper documentation on
>> what is allowed and what is not, also access to logs makes it impossible to
>> debug things on the IDP side. There is some interesting discussion about it
>> on the Azure forum [1]. I think we need to take a good look at Azure AD
>> implementation.
>>
>>
>> [1] https://feedback.azure.com/forums/169401-azure-active-
>> directory/suggestions/6535001-openidconnect-bug-in-azure-ad-sso-reply-url
>> [2] http://www.pac4j.org/docs/config.html
>>
>> Best,
>> Sandeep
>>
>> On Thu, Feb 22, 2018 at 5:30 AM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> To summarise, it looks like there are three issues here (Correct me if
>>> I'm wrong):
>>>
>>> a) There is a regression in 1.0.0 due to the Pac4J upgrade when doing
>>> OIDC with Google. It's resolved by switching to use the J2ESessionStore.
>>> b) Doing OIDC (at least in Knox 0.12.0) with Azure does not work.
>>> Possibly because Azure is not returning the pac4jCallback query parameter?
>>> c) Pac4j should not redirect back to the IdP if there is an error
>>> processing the initial response, but instead return an error to the user.
>>>
>>> Colm.
>>>
>>> On Wed, Feb 21, 2018 at 2:01 PM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hi Sandeep,
>>>>
>>>> Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
>>>> thought it was internally understood.
>>>> Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does
>>>> not work for you, it may not work for me as well.
>>>>
>>>> Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt
>>>> work for me since my version was KNOX 0.12, will it work for higher
>>>> versions?
>>>>
>>>> Regards
>>>> Nisha
>>>>
>>>> On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Nisha,
>>>>>
>>>>> Yes, you can get the source code from git '
>>>>> https://github.com/apache/knox'.
>>>>> I found changing to "J2ESessionStore" worked for Google OpenID and
>>>>> not for Azure, let me know if you are looking for the patch for this change
>>>>> I can share it with you.
>>>>>
>>>>> I did try "J2ESessionStore" with Azure AD but I am running into the
>>>>> issues you are seeing, multiple redirects.
>>>>> Interestingly enough I see that callback url from Azure is something
>>>>> like this
>>>>> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr
>>>>> ......."
>>>>>
>>>>> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
>>>>> looking for, I will investigate this more today and let you know.
>>>>>
>>>>> Best,
>>>>> Sandeep
>>>>>
>>>>>
>>>>> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <nisha.menon16@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Hi Colm/Larry,
>>>>>>
>>>>>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the
>>>>>> full HDP 2.6 stack. So the bundled version is KNOX 0.12.
>>>>>> The upgrades supported in HDP are w.r.t corresponding stack version
>>>>>> only I guess. Since 2.6 is the highest version (for HDP stack) available, I
>>>>>> dont know if KNOX can be seperately upgraded to any version higher than
>>>>>> 0.12.
>>>>>>
>>>>>> Colm/Sandeep: Could you please detail the KnoxSessionStore change
>>>>>> steps little more? Where do I find the source code? Do I need to get the
>>>>>> whole source code from git, change and build it again?
>>>>>>
>>>>>> These are the pac4j jars available in my stack now:
>>>>>> [root@localhost topologies]# locate pac4j
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>>>>>> /usr/hdp/2.6.1.0-129/knox/lib/
>>>>>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>>>>>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>>>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pa
>>>>>> c4j-credentials.jceks
>>>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>>>>>> pac4j-credentials.jceks
>>>>>>
>>>>>> Would it work if I replace the pac4j oidc and gateway provider jar
>>>>>> with latest version?
>>>>>>
>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>>>>>> successfully with 0.13.0, so maybe we are running into different issues.
>>>>>>> Could you try Azure with 0.13.0 and see if it works?
>>>>>>>
>>>>>>> @Larry, switching to use GoogleOidcClient doesn't make any
>>>>>>> difference. I just changed KnoxSessionStore manually and copied it over
>>>>>>> into my distribution :-)
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Oh, bummer.
>>>>>>>>
>>>>>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>>>>>> J2ESessionStore?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <
>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Larry,
>>>>>>>>>
>>>>>>>>> Well, this is what I started with in my implementation. I tested
>>>>>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>>>>>> seem to be recongnised in pac4j client.
>>>>>>>>> Thats when I used "OidcClient" itself".
>>>>>>>>> Error:
>>>>>>>>>
>>>>>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>>>>>> found for name: AzureAdClient
>>>>>>>>> at org.pac4j.core.client.Clients.
>>>>>>>>> findClient(Clients.java:147)
>>>>>>>>> at org.pac4j.core.client.DefaultC
>>>>>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Nisha
>>>>>>>>>
>>>>>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and
>>>>>>>>>> Google are not.
>>>>>>>>>> While I've never tested AAD, I have tested Google so there may be
>>>>>>>>>> a regression there.
>>>>>>>>>>
>>>>>>>>>> I notice that there are specific clients for both AzureAd and
>>>>>>>>>> Google now - see: http://www.pac4j.org/docs
>>>>>>>>>> /clients/openid-connect.html#2-clients
>>>>>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>>>>>
>>>>>>>>>> They also have corresponding profile creators.
>>>>>>>>>>
>>>>>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient
>>>>>>>>>> and AzureAdOidcClient ?
>>>>>>>>>>
>>>>>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Trying with "www.local.com" makes no difference for me. The
>>>>>>>>>>> problem is that it is not retrieving the stored user profile correctly
>>>>>>>>>>> after it redirects back to the Pac4jDispatcherFilter:
>>>>>>>>>>>
>>>>>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>>>>>> (DefaultSecurityLogic.java:perform(98)) -
>>>>>>>>>>> loadProfilesFromSession: true
>>>>>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>> pac4jUserProfiles = null
>>>>>>>>>>>
>>>>>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>>>>>
>>>>>>>>>>> and it worked! Perhaps we should change from storing the profile
>>>>>>>>>>> in a cookie to an attribute on the session instead?
>>>>>>>>>>>
>>>>>>>>>>> Colm.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>>>>>> originally requested resource.
>>>>>>>>>>>>
>>>>>>>>>>>> The KnoxSSO service assumes that by the time the request gets
>>>>>>>>>>>> to it that authentication/federation of the enduser has been done, that the
>>>>>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>>>>>
>>>>>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>>>>>
>>>>>>>>>>>> In this case, SSOCookieProvider will be then looking for the
>>>>>>>>>>>> cookie and will redirect back to KnoxSSO if it is not present.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> OK I'll check it. A question though - where is the redirection
>>>>>>>>>>>>> via originalUrl supposed to take place? From the source I can see
>>>>>>>>>>>>> "originalUrl" is referenced in:
>>>>>>>>>>>>>
>>>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Colm.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <
>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Colm -
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I see that you are using localhost - that will definitely
>>>>>>>>>>>>>> present problems for Set-Cookie with some browsers.
>>>>>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>>>>>> I usually just map localhost to www.local.com in my
>>>>>>>>>>>>>> /etc/hosts file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>>>>>> domain name.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --larry
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see
>>>>>>>>>>>>>>> the following:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>>>> PEN
>>>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>>>> PEN
>>>>>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It is getting an access token correctly and trying to
>>>>>>>>>>>>>>> redirect back to the original URL. However from the logs above it seems to
>>>>>>>>>>>>>>> never hit the original URL but instead hits the "redirectUrl". Is some
>>>>>>>>>>>>>>> parsing supposed to take place to extract "originalUrl" from the
>>>>>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Colm.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <
>>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting
>>>>>>>>>>>>>>>> past the pac4j federation provider to the KnoxSSO service itself because
>>>>>>>>>>>>>>>> there is a redirect from the pac4j provider or the Set-Cookie just isn't
>>>>>>>>>>>>>>>> being accepted by the browser.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are
>>>>>>>>>>>>>>>> in unchartered waters here.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and
>>>>>>>>>>>>>>>>> Azure AD.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a
>>>>>>>>>>>>>>>>>> couple things:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually
>>>>>>>>>>>>>>>>>> set in your browser
>>>>>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think that ip address should work for this case
>>>>>>>>>>>>>>>>>> actually but there are differences in browsers that might not let it.
>>>>>>>>>>>>>>>>>> Also, if you had another service on another ip address,
>>>>>>>>>>>>>>>>>> the browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also,
>>>>>>>>>>>>>>>>>>> can you turn up the logs to debug and share them along with the audit log
>>>>>>>>>>>>>>>>>>> that would help us to understand the problem better.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, after the authentication at Azure login page,
>>>>>>>>>>>>>>>>>>>> it gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it
>>>>>>>>>>>>>>>>>>>> redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling
>>>>>>>>>>>>>>>>>>>> nonces, knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing
>>>>>>>>>>>>>>>>>>>> helps.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not
>>>>>>>>>>>>>>>>>>>> "200". This leads to redirection!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be
>>>>>>>>>>>>>>>>>>>> greatly appreciated.
>>>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>>
>>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>
>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
All -
This is an unfortunate situation but a great triage effort!
I think this could be captured in a KIP as Sandeep suggests - especially if
it requires the communication of some design or alternative approaches.
It may also be sufficient to track multiple issues with an umbrella JIRA if
this seems like more of a list of bugs and improvements.
I have been a bit delayed in getting out a planning thread for 1.1.0 but I
do think that these issues need to addressed there.
Especially the regression for Google.
Azure AD is a new pac4j feature at least which we hadn't ever tried before.
SAML based integration with aad may still actually work.
thanks!
--larry
On Thu, Feb 22, 2018 at 9:15 AM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Colm,
>
> Thanks for summarizing, you are right with a) and b)
> about c) though I think it is doing the right thing, the issue I saw in my
> debugging was that the callback filter for Pac4J was never hit (absence of
> callback param), as a result the request from IDP would go to security
> filter and then get redirected to IDP, IDP had already authenticated so it
> redirects back to Knox, hence the loop.
>
> I would like to add few things to your list that I found out:
>
> d) Support for "oidc.type" [2], this is the property that chooses Azure
> or Google client, although not sure how much it will help.
> e) Config variables "defaultUrl", "multiProfile" and "renewSession" are
> not getting picked up.
>
> I think we need a separate KIP to track all these issues. On the face of
> it, a) seems easy to fix and we can get folks unblocked asap.
>
> I really hate that Azure does not honor the callback parameters, it is
> frustrating trying to debug the problem without a proper documentation on
> what is allowed and what is not, also access to logs makes it impossible to
> debug things on the IDP side. There is some interesting discussion about it
> on the Azure forum [1]. I think we need to take a good look at Azure AD
> implementation.
>
>
> [1] https://feedback.azure.com/forums/169401-azure-
> active-directory/suggestions/6535001-openidconnect-bug-in-
> azure-ad-sso-reply-url
> [2] http://www.pac4j.org/docs/config.html
>
> Best,
> Sandeep
>
> On Thu, Feb 22, 2018 at 5:30 AM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
>> To summarise, it looks like there are three issues here (Correct me if
>> I'm wrong):
>>
>> a) There is a regression in 1.0.0 due to the Pac4J upgrade when doing
>> OIDC with Google. It's resolved by switching to use the J2ESessionStore.
>> b) Doing OIDC (at least in Knox 0.12.0) with Azure does not work.
>> Possibly because Azure is not returning the pac4jCallback query parameter?
>> c) Pac4j should not redirect back to the IdP if there is an error
>> processing the initial response, but instead return an error to the user.
>>
>> Colm.
>>
>> On Wed, Feb 21, 2018 at 2:01 PM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hi Sandeep,
>>>
>>> Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
>>> thought it was internally understood.
>>> Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does
>>> not work for you, it may not work for me as well.
>>>
>>> Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt
>>> work for me since my version was KNOX 0.12, will it work for higher
>>> versions?
>>>
>>> Regards
>>> Nisha
>>>
>>> On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> Hello Nisha,
>>>>
>>>> Yes, you can get the source code from git '
>>>> https://github.com/apache/knox'.
>>>> I found changing to "J2ESessionStore" worked for Google OpenID and not
>>>> for Azure, let me know if you are looking for the patch for this change I
>>>> can share it with you.
>>>>
>>>> I did try "J2ESessionStore" with Azure AD but I am running into the
>>>> issues you are seeing, multiple redirects.
>>>> Interestingly enough I see that callback url from Azure is something
>>>> like this
>>>> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr ......."
>>>>
>>>> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
>>>> looking for, I will investigate this more today and let you know.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>>
>>>> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Colm/Larry,
>>>>>
>>>>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the
>>>>> full HDP 2.6 stack. So the bundled version is KNOX 0.12.
>>>>> The upgrades supported in HDP are w.r.t corresponding stack version
>>>>> only I guess. Since 2.6 is the highest version (for HDP stack) available, I
>>>>> dont know if KNOX can be seperately upgraded to any version higher than
>>>>> 0.12.
>>>>>
>>>>> Colm/Sandeep: Could you please detail the KnoxSessionStore change
>>>>> steps little more? Where do I find the source code? Do I need to get the
>>>>> whole source code from git, change and build it again?
>>>>>
>>>>> These are the pac4j jars available in my stack now:
>>>>> [root@localhost topologies]# locate pac4j
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>>>>> /usr/hdp/2.6.1.0-129/knox/lib/
>>>>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>>>>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pa
>>>>> c4j-credentials.jceks
>>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>>>>> pac4j-credentials.jceks
>>>>>
>>>>> Would it work if I replace the pac4j oidc and gateway provider jar
>>>>> with latest version?
>>>>>
>>>>> Regards
>>>>> Nisha
>>>>>
>>>>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <
>>>>> coheigea@apache.org> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>>>>> successfully with 0.13.0, so maybe we are running into different issues.
>>>>>> Could you try Azure with 0.13.0 and see if it works?
>>>>>>
>>>>>> @Larry, switching to use GoogleOidcClient doesn't make any
>>>>>> difference. I just changed KnoxSessionStore manually and copied it over
>>>>>> into my distribution :-)
>>>>>>
>>>>>> Colm.
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Oh, bummer.
>>>>>>>
>>>>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>>>>> J2ESessionStore?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <
>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello Larry,
>>>>>>>>
>>>>>>>> Well, this is what I started with in my implementation. I tested
>>>>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>>>>> seem to be recongnised in pac4j client.
>>>>>>>> Thats when I used "OidcClient" itself".
>>>>>>>> Error:
>>>>>>>>
>>>>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>>>>> found for name: AzureAdClient
>>>>>>>> at org.pac4j.core.client.Clients.
>>>>>>>> findClient(Clients.java:147)
>>>>>>>> at org.pac4j.core.client.DefaultC
>>>>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Nisha
>>>>>>>>
>>>>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and
>>>>>>>>> Google are not.
>>>>>>>>> While I've never tested AAD, I have tested Google so there may be
>>>>>>>>> a regression there.
>>>>>>>>>
>>>>>>>>> I notice that there are specific clients for both AzureAd and
>>>>>>>>> Google now - see: http://www.pac4j.org/docs
>>>>>>>>> /clients/openid-connect.html#2-clients
>>>>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>>>>
>>>>>>>>> They also have corresponding profile creators.
>>>>>>>>>
>>>>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>>>>>>> AzureAdOidcClient ?
>>>>>>>>>
>>>>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Trying with "www.local.com" makes no difference for me. The
>>>>>>>>>> problem is that it is not retrieving the stored user profile correctly
>>>>>>>>>> after it redirects back to the Pac4jDispatcherFilter:
>>>>>>>>>>
>>>>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>>>>> (DefaultSecurityLogic.java:perform(98)) -
>>>>>>>>>> loadProfilesFromSession: true
>>>>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>> pac4jUserProfiles = null
>>>>>>>>>>
>>>>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>>>>
>>>>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>>>>
>>>>>>>>>> and it worked! Perhaps we should change from storing the profile
>>>>>>>>>> in a cookie to an attribute on the session instead?
>>>>>>>>>>
>>>>>>>>>> Colm.
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>>>>> originally requested resource.
>>>>>>>>>>>
>>>>>>>>>>> The KnoxSSO service assumes that by the time the request gets to
>>>>>>>>>>> it that authentication/federation of the enduser has been done, that the
>>>>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>>>>
>>>>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>>>>
>>>>>>>>>>> In this case, SSOCookieProvider will be then looking for the
>>>>>>>>>>> cookie and will redirect back to KnoxSSO if it is not present.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> OK I'll check it. A question though - where is the redirection
>>>>>>>>>>>> via originalUrl supposed to take place? From the source I can see
>>>>>>>>>>>> "originalUrl" is referenced in:
>>>>>>>>>>>>
>>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>>>>
>>>>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>>>>
>>>>>>>>>>>> Colm.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lmccay@apache.org
>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Colm -
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>>>>
>>>>>>>>>>>>> I see that you are using localhost - that will definitely
>>>>>>>>>>>>> present problems for Set-Cookie with some browsers.
>>>>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>>>>> I usually just map localhost to www.local.com in my
>>>>>>>>>>>>> /etc/hosts file.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>>>>> domain name.
>>>>>>>>>>>>>
>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> --larry
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see
>>>>>>>>>>>>>> the following:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>>> PEN
>>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>>> PEN
>>>>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is getting an access token correctly and trying to
>>>>>>>>>>>>>> redirect back to the original URL. However from the logs above it seems to
>>>>>>>>>>>>>> never hit the original URL but instead hits the "redirectUrl". Is some
>>>>>>>>>>>>>> parsing supposed to take place to extract "originalUrl" from the
>>>>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Colm.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <
>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting
>>>>>>>>>>>>>>> past the pac4j federation provider to the KnoxSSO service itself because
>>>>>>>>>>>>>>> there is a redirect from the pac4j provider or the Set-Cookie just isn't
>>>>>>>>>>>>>>> being accepted by the browser.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>>>>>>> unchartered waters here.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and
>>>>>>>>>>>>>>>> Azure AD.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a
>>>>>>>>>>>>>>>>> couple things:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually
>>>>>>>>>>>>>>>>> set in your browser
>>>>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think that ip address should work for this case actually
>>>>>>>>>>>>>>>>> but there are differences in browsers that might not let it.
>>>>>>>>>>>>>>>>> Also, if you had another service on another ip address,
>>>>>>>>>>>>>>>>> the browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can
>>>>>>>>>>>>>>>>>> you turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> However, after the authentication at Azure login page,
>>>>>>>>>>>>>>>>>>> it gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it
>>>>>>>>>>>>>>>>>>> redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200".
>>>>>>>>>>>>>>>>>>> This leads to redirection!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be
>>>>>>>>>>>>>>>>>>> greatly appreciated.
>>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>
>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>
>>>>>>>>>> Talend Community Coder
>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Colm O hEigeartaigh
>>>>>>
>>>>>> Talend Community Coder
>>>>>> http://coders.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Colm,
Thanks for summarizing, you are right with a) and b)
about c) though I think it is doing the right thing, the issue I saw in my
debugging was that the callback filter for Pac4J was never hit (absence of
callback param), as a result the request from IDP would go to security
filter and then get redirected to IDP, IDP had already authenticated so it
redirects back to Knox, hence the loop.
I would like to add few things to your list that I found out:
d) Support for "oidc.type" [2], this is the property that chooses Azure or
Google client, although not sure how much it will help.
e) Config variables "defaultUrl", "multiProfile" and "renewSession" are
not getting picked up.
I think we need a separate KIP to track all these issues. On the face of
it, a) seems easy to fix and we can get folks unblocked asap.
I really hate that Azure does not honor the callback parameters, it is
frustrating trying to debug the problem without a proper documentation on
what is allowed and what is not, also access to logs makes it impossible to
debug things on the IDP side. There is some interesting discussion about it
on the Azure forum [1]. I think we need to take a good look at Azure AD
implementation.
[1]
https://feedback.azure.com/forums/169401-azure-active-directory/suggestions/6535001-openidconnect-bug-in-azure-ad-sso-reply-url
[2] http://www.pac4j.org/docs/config.html
Best,
Sandeep
On Thu, Feb 22, 2018 at 5:30 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> To summarise, it looks like there are three issues here (Correct me if I'm
> wrong):
>
> a) There is a regression in 1.0.0 due to the Pac4J upgrade when doing OIDC
> with Google. It's resolved by switching to use the J2ESessionStore.
> b) Doing OIDC (at least in Knox 0.12.0) with Azure does not work. Possibly
> because Azure is not returning the pac4jCallback query parameter?
> c) Pac4j should not redirect back to the IdP if there is an error
> processing the initial response, but instead return an error to the user.
>
> Colm.
>
> On Wed, Feb 21, 2018 at 2:01 PM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hi Sandeep,
>>
>> Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
>> thought it was internally understood.
>> Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does
>> not work for you, it may not work for me as well.
>>
>> Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt
>> work for me since my version was KNOX 0.12, will it work for higher
>> versions?
>>
>> Regards
>> Nisha
>>
>> On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> Hello Nisha,
>>>
>>> Yes, you can get the source code from git 'https://github.com/apache/kno
>>> x'.
>>> I found changing to "J2ESessionStore" worked for Google OpenID and not
>>> for Azure, let me know if you are looking for the patch for this change I
>>> can share it with you.
>>>
>>> I did try "J2ESessionStore" with Azure AD but I am running into the
>>> issues you are seeing, multiple redirects.
>>> Interestingly enough I see that callback url from Azure is something
>>> like this
>>> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr ......."
>>>
>>> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
>>> looking for, I will investigate this more today and let you know.
>>>
>>> Best,
>>> Sandeep
>>>
>>>
>>> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hi Colm/Larry,
>>>>
>>>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the
>>>> full HDP 2.6 stack. So the bundled version is KNOX 0.12.
>>>> The upgrades supported in HDP are w.r.t corresponding stack version
>>>> only I guess. Since 2.6 is the highest version (for HDP stack) available, I
>>>> dont know if KNOX can be seperately upgraded to any version higher than
>>>> 0.12.
>>>>
>>>> Colm/Sandeep: Could you please detail the KnoxSessionStore change steps
>>>> little more? Where do I find the source code? Do I need to get the whole
>>>> source code from git, change and build it again?
>>>>
>>>> These are the pac4j jars available in my stack now:
>>>> [root@localhost topologies]# locate pac4j
>>>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/lib/
>>>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>>>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pa
>>>> c4j-credentials.jceks
>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>>>> pac4j-credentials.jceks
>>>>
>>>> Would it work if I replace the pac4j oidc and gateway provider jar with
>>>> latest version?
>>>>
>>>> Regards
>>>> Nisha
>>>>
>>>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>>>> successfully with 0.13.0, so maybe we are running into different issues.
>>>>> Could you try Azure with 0.13.0 and see if it works?
>>>>>
>>>>> @Larry, switching to use GoogleOidcClient doesn't make any difference.
>>>>> I just changed KnoxSessionStore manually and copied it over into my
>>>>> distribution :-)
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Oh, bummer.
>>>>>>
>>>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>>>> J2ESessionStore?
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> Well, this is what I started with in my implementation. I tested
>>>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>>>> seem to be recongnised in pac4j client.
>>>>>>> Thats when I used "OidcClient" itself".
>>>>>>> Error:
>>>>>>>
>>>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>>>> found for name: AzureAdClient
>>>>>>> at org.pac4j.core.client.Clients.
>>>>>>> findClient(Clients.java:147)
>>>>>>> at org.pac4j.core.client.DefaultC
>>>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>>>
>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and
>>>>>>>> Google are not.
>>>>>>>> While I've never tested AAD, I have tested Google so there may be a
>>>>>>>> regression there.
>>>>>>>>
>>>>>>>> I notice that there are specific clients for both AzureAd and
>>>>>>>> Google now - see: http://www.pac4j.org/docs
>>>>>>>> /clients/openid-connect.html#2-clients
>>>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>>>
>>>>>>>> They also have corresponding profile creators.
>>>>>>>>
>>>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>>>>>> AzureAdOidcClient ?
>>>>>>>>
>>>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Trying with "www.local.com" makes no difference for me. The
>>>>>>>>> problem is that it is not retrieving the stored user profile correctly
>>>>>>>>> after it redirects back to the Pac4jDispatcherFilter:
>>>>>>>>>
>>>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>>>> (DefaultSecurityLogic.java:perform(98)) -
>>>>>>>>> loadProfilesFromSession: true
>>>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>> pac4jUserProfiles = null
>>>>>>>>>
>>>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>>>
>>>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>>>
>>>>>>>>> and it worked! Perhaps we should change from storing the profile
>>>>>>>>> in a cookie to an attribute on the session instead?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>>>> originally requested resource.
>>>>>>>>>>
>>>>>>>>>> The KnoxSSO service assumes that by the time the request gets to
>>>>>>>>>> it that authentication/federation of the enduser has been done, that the
>>>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>>>
>>>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>>>
>>>>>>>>>> In this case, SSOCookieProvider will be then looking for the
>>>>>>>>>> cookie and will redirect back to KnoxSSO if it is not present.
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> OK I'll check it. A question though - where is the redirection
>>>>>>>>>>> via originalUrl supposed to take place? From the source I can see
>>>>>>>>>>> "originalUrl" is referenced in:
>>>>>>>>>>>
>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>>>
>>>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>>>
>>>>>>>>>>> Colm.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Colm -
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>>>
>>>>>>>>>>>> I see that you are using localhost - that will definitely
>>>>>>>>>>>> present problems for Set-Cookie with some browsers.
>>>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>>>
>>>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>>>>>> file.
>>>>>>>>>>>>
>>>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>>>> domain name.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> --larry
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>>>>>> following:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>> PEN
>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>> PEN
>>>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>>>
>>>>>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Colm.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <
>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting past
>>>>>>>>>>>>>> the pac4j federation provider to the KnoxSSO service itself because there
>>>>>>>>>>>>>> is a redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>>>>>> accepted by the browser.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>>>>>> unchartered waters here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure
>>>>>>>>>>>>>>> AD.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a
>>>>>>>>>>>>>>>> couple things:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set
>>>>>>>>>>>>>>>> in your browser
>>>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think that ip address should work for this case actually
>>>>>>>>>>>>>>>> but there are differences in browsers that might not let it.
>>>>>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can
>>>>>>>>>>>>>>>>> you turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it
>>>>>>>>>>>>>>>>>> redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200".
>>>>>>>>>>>>>>>>>> This leads to redirection!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be
>>>>>>>>>>>>>>>>>> greatly appreciated.
>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>>
>>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>
>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hello Colm ,
I tested with KNOX 1.0.0 as well with Azure AD/OidcClient, it also has the
same issues.
On Thu, Feb 22, 2018 at 4:00 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> To summarise, it looks like there are three issues here (Correct me if I'm
> wrong):
>
> a) There is a regression in 1.0.0 due to the Pac4J upgrade when doing OIDC
> with Google. It's resolved by switching to use the J2ESessionStore.
> b) Doing OIDC (at least in Knox 0.12.0) with Azure does not work. Possibly
> because Azure is not returning the pac4jCallback query parameter?
> c) Pac4j should not redirect back to the IdP if there is an error
> processing the initial response, but instead return an error to the user.
>
> Colm.
>
> On Wed, Feb 21, 2018 at 2:01 PM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hi Sandeep,
>>
>> Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
>> thought it was internally understood.
>> Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does
>> not work for you, it may not work for me as well.
>>
>> Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt
>> work for me since my version was KNOX 0.12, will it work for higher
>> versions?
>>
>> Regards
>> Nisha
>>
>> On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> Hello Nisha,
>>>
>>> Yes, you can get the source code from git 'https://github.com/apache/kno
>>> x'.
>>> I found changing to "J2ESessionStore" worked for Google OpenID and not
>>> for Azure, let me know if you are looking for the patch for this change I
>>> can share it with you.
>>>
>>> I did try "J2ESessionStore" with Azure AD but I am running into the
>>> issues you are seeing, multiple redirects.
>>> Interestingly enough I see that callback url from Azure is something
>>> like this
>>> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr ......."
>>>
>>> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
>>> looking for, I will investigate this more today and let you know.
>>>
>>> Best,
>>> Sandeep
>>>
>>>
>>> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hi Colm/Larry,
>>>>
>>>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the
>>>> full HDP 2.6 stack. So the bundled version is KNOX 0.12.
>>>> The upgrades supported in HDP are w.r.t corresponding stack version
>>>> only I guess. Since 2.6 is the highest version (for HDP stack) available, I
>>>> dont know if KNOX can be seperately upgraded to any version higher than
>>>> 0.12.
>>>>
>>>> Colm/Sandeep: Could you please detail the KnoxSessionStore change steps
>>>> little more? Where do I find the source code? Do I need to get the whole
>>>> source code from git, change and build it again?
>>>>
>>>> These are the pac4j jars available in my stack now:
>>>> [root@localhost topologies]# locate pac4j
>>>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>>>> /usr/hdp/2.6.1.0-129/knox/lib/
>>>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>>>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pa
>>>> c4j-credentials.jceks
>>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>>>> pac4j-credentials.jceks
>>>>
>>>> Would it work if I replace the pac4j oidc and gateway provider jar with
>>>> latest version?
>>>>
>>>> Regards
>>>> Nisha
>>>>
>>>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>>>> successfully with 0.13.0, so maybe we are running into different issues.
>>>>> Could you try Azure with 0.13.0 and see if it works?
>>>>>
>>>>> @Larry, switching to use GoogleOidcClient doesn't make any difference.
>>>>> I just changed KnoxSessionStore manually and copied it over into my
>>>>> distribution :-)
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Oh, bummer.
>>>>>>
>>>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>>>> J2ESessionStore?
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> Well, this is what I started with in my implementation. I tested
>>>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>>>> seem to be recongnised in pac4j client.
>>>>>>> Thats when I used "OidcClient" itself".
>>>>>>> Error:
>>>>>>>
>>>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>>>> found for name: AzureAdClient
>>>>>>> at org.pac4j.core.client.Clients.
>>>>>>> findClient(Clients.java:147)
>>>>>>> at org.pac4j.core.client.DefaultC
>>>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>>>
>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and
>>>>>>>> Google are not.
>>>>>>>> While I've never tested AAD, I have tested Google so there may be a
>>>>>>>> regression there.
>>>>>>>>
>>>>>>>> I notice that there are specific clients for both AzureAd and
>>>>>>>> Google now - see: http://www.pac4j.org/docs
>>>>>>>> /clients/openid-connect.html#2-clients
>>>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>>>
>>>>>>>> They also have corresponding profile creators.
>>>>>>>>
>>>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>>>>>> AzureAdOidcClient ?
>>>>>>>>
>>>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Trying with "www.local.com" makes no difference for me. The
>>>>>>>>> problem is that it is not retrieving the stored user profile correctly
>>>>>>>>> after it redirects back to the Pac4jDispatcherFilter:
>>>>>>>>>
>>>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>>>> (DefaultSecurityLogic.java:perform(98)) -
>>>>>>>>> loadProfilesFromSession: true
>>>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>> pac4jUserProfiles = null
>>>>>>>>>
>>>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>>>
>>>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>>>
>>>>>>>>> and it worked! Perhaps we should change from storing the profile
>>>>>>>>> in a cookie to an attribute on the session instead?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>>>> originally requested resource.
>>>>>>>>>>
>>>>>>>>>> The KnoxSSO service assumes that by the time the request gets to
>>>>>>>>>> it that authentication/federation of the enduser has been done, that the
>>>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>>>
>>>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>>>
>>>>>>>>>> In this case, SSOCookieProvider will be then looking for the
>>>>>>>>>> cookie and will redirect back to KnoxSSO if it is not present.
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> OK I'll check it. A question though - where is the redirection
>>>>>>>>>>> via originalUrl supposed to take place? From the source I can see
>>>>>>>>>>> "originalUrl" is referenced in:
>>>>>>>>>>>
>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>>>
>>>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>>>
>>>>>>>>>>> Colm.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Colm -
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>>>
>>>>>>>>>>>> I see that you are using localhost - that will definitely
>>>>>>>>>>>> present problems for Set-Cookie with some browsers.
>>>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>>>
>>>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>>>>>> file.
>>>>>>>>>>>>
>>>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>>>> domain name.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> --larry
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>>>>>> following:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>> PEN
>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=O
>>>>>>>>>>>>> PEN
>>>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>>>
>>>>>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Colm.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <
>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting past
>>>>>>>>>>>>>> the pac4j federation provider to the KnoxSSO service itself because there
>>>>>>>>>>>>>> is a redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>>>>>> accepted by the browser.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>>>>>> unchartered waters here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure
>>>>>>>>>>>>>>> AD.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a
>>>>>>>>>>>>>>>> couple things:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set
>>>>>>>>>>>>>>>> in your browser
>>>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think that ip address should work for this case actually
>>>>>>>>>>>>>>>> but there are differences in browsers that might not let it.
>>>>>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can
>>>>>>>>>>>>>>>>> you turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it
>>>>>>>>>>>>>>>>>> redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200".
>>>>>>>>>>>>>>>>>> This leads to redirection!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be
>>>>>>>>>>>>>>>>>> greatly appreciated.
>>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>>
>>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>
>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Colm O hEigeartaigh <co...@apache.org>.
To summarise, it looks like there are three issues here (Correct me if I'm
wrong):
a) There is a regression in 1.0.0 due to the Pac4J upgrade when doing OIDC
with Google. It's resolved by switching to use the J2ESessionStore.
b) Doing OIDC (at least in Knox 0.12.0) with Azure does not work. Possibly
because Azure is not returning the pac4jCallback query parameter?
c) Pac4j should not redirect back to the IdP if there is an error
processing the initial response, but instead return an error to the user.
Colm.
On Wed, Feb 21, 2018 at 2:01 PM, Nisha Menon <ni...@gmail.com>
wrote:
> Hi Sandeep,
>
> Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
> thought it was internally understood.
> Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does not
> work for you, it may not work for me as well.
>
> Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt
> work for me since my version was KNOX 0.12, will it work for higher
> versions?
>
> Regards
> Nisha
>
> On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Nisha,
>>
>> Yes, you can get the source code from git 'https://github.com/apache/knox
>> '.
>> I found changing to "J2ESessionStore" worked for Google OpenID and not
>> for Azure, let me know if you are looking for the patch for this change I
>> can share it with you.
>>
>> I did try "J2ESessionStore" with Azure AD but I am running into the
>> issues you are seeing, multiple redirects.
>> Interestingly enough I see that callback url from Azure is something like
>> this
>> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr ......."
>>
>> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
>> looking for, I will investigate this more today and let you know.
>>
>> Best,
>> Sandeep
>>
>>
>> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hi Colm/Larry,
>>>
>>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the full
>>> HDP 2.6 stack. So the bundled version is KNOX 0.12.
>>> The upgrades supported in HDP are w.r.t corresponding stack version only
>>> I guess. Since 2.6 is the highest version (for HDP stack) available, I dont
>>> know if KNOX can be seperately upgraded to any version higher than 0.12.
>>>
>>> Colm/Sandeep: Could you please detail the KnoxSessionStore change steps
>>> little more? Where do I find the source code? Do I need to get the whole
>>> source code from git, change and build it again?
>>>
>>> These are the pac4j jars available in my stack now:
>>> [root@localhost topologies]# locate pac4j
>>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>>> /usr/hdp/2.6.1.0-129/knox/lib/
>>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pa
>>> c4j-credentials.jceks
>>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>>> pac4j-credentials.jceks
>>>
>>> Would it work if I replace the pac4j oidc and gateway provider jar with
>>> latest version?
>>>
>>> Regards
>>> Nisha
>>>
>>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> Hi all,
>>>>
>>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>>> successfully with 0.13.0, so maybe we are running into different issues.
>>>> Could you try Azure with 0.13.0 and see if it works?
>>>>
>>>> @Larry, switching to use GoogleOidcClient doesn't make any difference.
>>>> I just changed KnoxSessionStore manually and copied it over into my
>>>> distribution :-)
>>>>
>>>> Colm.
>>>>
>>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> Oh, bummer.
>>>>>
>>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>>> J2ESessionStore?
>>>>>
>>>>>
>>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Larry,
>>>>>>
>>>>>> Well, this is what I started with in my implementation. I tested
>>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>>> seem to be recongnised in pac4j client.
>>>>>> Thats when I used "OidcClient" itself".
>>>>>> Error:
>>>>>>
>>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>>> found for name: AzureAdClient
>>>>>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>>>>>> at org.pac4j.core.client.DefaultC
>>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>>
>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and Google
>>>>>>> are not.
>>>>>>> While I've never tested AAD, I have tested Google so there may be a
>>>>>>> regression there.
>>>>>>>
>>>>>>> I notice that there are specific clients for both AzureAd and Google
>>>>>>> now - see: http://www.pac4j.org/docs/clients/openid-connect.html#2
>>>>>>> -clients
>>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>>
>>>>>>> They also have corresponding profile creators.
>>>>>>>
>>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>>>>> AzureAdOidcClient ?
>>>>>>>
>>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>>> coheigea@apache.org> wrote:
>>>>>>>
>>>>>>>> Trying with "www.local.com" makes no difference for me. The
>>>>>>>> problem is that it is not retrieving the stored user profile correctly
>>>>>>>> after it redirects back to the Pac4jDispatcherFilter:
>>>>>>>>
>>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession:
>>>>>>>> true
>>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>> pac4jUserProfiles = null
>>>>>>>>
>>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>>
>>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>>
>>>>>>>> and it worked! Perhaps we should change from storing the profile in
>>>>>>>> a cookie to an attribute on the session instead?
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>>> originally requested resource.
>>>>>>>>>
>>>>>>>>> The KnoxSSO service assumes that by the time the request gets to
>>>>>>>>> it that authentication/federation of the enduser has been done, that the
>>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>>
>>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>>
>>>>>>>>> In this case, SSOCookieProvider will be then looking for the
>>>>>>>>> cookie and will redirect back to KnoxSSO if it is not present.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> OK I'll check it. A question though - where is the redirection
>>>>>>>>>> via originalUrl supposed to take place? From the source I can see
>>>>>>>>>> "originalUrl" is referenced in:
>>>>>>>>>>
>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>>
>>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>>
>>>>>>>>>> Colm.
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Colm -
>>>>>>>>>>>
>>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>>
>>>>>>>>>>> I see that you are using localhost - that will definitely
>>>>>>>>>>> present problems for Set-Cookie with some browsers.
>>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>>
>>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>>>>> file.
>>>>>>>>>>>
>>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>>> domain name.
>>>>>>>>>>>
>>>>>>>>>>> thanks,
>>>>>>>>>>>
>>>>>>>>>>> --larry
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>>>>> following:
>>>>>>>>>>>>
>>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>>
>>>>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>>
>>>>>>>>>>>> Colm.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lmccay@apache.org
>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting past
>>>>>>>>>>>>> the pac4j federation provider to the KnoxSSO service itself because there
>>>>>>>>>>>>> is a redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>>>>> accepted by the browser.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>>>>> unchartered waters here.
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure
>>>>>>>>>>>>>> AD.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a
>>>>>>>>>>>>>>> couple things:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set
>>>>>>>>>>>>>>> in your browser
>>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think that ip address should work for this case actually
>>>>>>>>>>>>>>> but there are differences in browsers that might not let it.
>>>>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can
>>>>>>>>>>>>>>>> you turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it
>>>>>>>>>>>>>>>>> redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200".
>>>>>>>>>>>>>>>>> This leads to redirection!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>>
>>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>
>>>>>>>>>> Talend Community Coder
>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Colm O hEigeartaigh
>>>>>>>>
>>>>>>>> Talend Community Coder
>>>>>>>> http://coders.talend.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hi Sandeep,
Yes, I did notice that 'pac4jCallback=true' is missing in that URL. I
thought it was internally understood.
Yeah, I am looking for a patch for AzureAD. So if J2ESessionStore does not
work for you, it may not work for me as well.
Did you also confirm "AzureAdClient" instead of "OidcClient"? It didnt work
for me since my version was KNOX 0.12, will it work for higher versions?
Regards
Nisha
On Wed, Feb 21, 2018 at 7:22 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Nisha,
>
> Yes, you can get the source code from git 'https://github.com/apache/knox
> '.
> I found changing to "J2ESessionStore" worked for Google OpenID and not
> for Azure, let me know if you are looking for the patch for this change I
> can share it with you.
>
> I did try "J2ESessionStore" with Azure AD but I am running into the
> issues you are seeing, multiple redirects.
> Interestingly enough I see that callback url from Azure is something like
> this
> "/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5Xr ......."
>
> Notice, there is no 'pac4jCallback=true', which is what the Pac4J is
> looking for, I will investigate this more today and let you know.
>
> Best,
> Sandeep
>
>
> On Tue, Feb 20, 2018 at 11:09 PM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hi Colm/Larry,
>>
>> I am not quite sure how to upgrade KNOX. I have setup KNOX with the full
>> HDP 2.6 stack. So the bundled version is KNOX 0.12.
>> The upgrades supported in HDP are w.r.t corresponding stack version only
>> I guess. Since 2.6 is the highest version (for HDP stack) available, I dont
>> know if KNOX can be seperately upgraded to any version higher than 0.12.
>>
>> Colm/Sandeep: Could you please detail the KnoxSessionStore change steps
>> little more? Where do I find the source code? Do I need to get the whole
>> source code from git, change and build it again?
>>
>> These are the pac4j jars available in my stack now:
>> [root@localhost topologies]# locate pac4j
>> /usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
>> /usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
>> /usr/hdp/2.6.1.0-129/knox/lib/
>> *gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
>> /usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_
>> pac4j-credentials.jceks
>> /var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_
>> pac4j-credentials.jceks
>>
>> Would it work if I replace the pac4j oidc and gateway provider jar with
>> latest version?
>>
>> Regards
>> Nisha
>>
>> On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> Hi all,
>>>
>>> @Nisha - I assumed you were on 1.0.0 as well. I tested Google
>>> successfully with 0.13.0, so maybe we are running into different issues.
>>> Could you try Azure with 0.13.0 and see if it works?
>>>
>>> @Larry, switching to use GoogleOidcClient doesn't make any difference. I
>>> just changed KnoxSessionStore manually and copied it over into my
>>> distribution :-)
>>>
>>> Colm.
>>>
>>> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> Oh, bummer.
>>>>
>>>> @Colm - can you share how you changed the KnoxSessionStore with the
>>>> J2ESessionStore?
>>>>
>>>>
>>>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Larry,
>>>>>
>>>>> Well, this is what I started with in my implementation. I tested
>>>>> again. Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not
>>>>> seem to be recongnised in pac4j client.
>>>>> Thats when I used "OidcClient" itself".
>>>>> Error:
>>>>>
>>>>> Caused by: org.pac4j.core.exception.TechnicalException: No client
>>>>> found for name: AzureAdClient
>>>>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>>>>> at org.pac4j.core.client.DefaultC
>>>>> lientFinder.find(DefaultClientFinder.java:56)
>>>>>
>>>>> Regards
>>>>> Nisha
>>>>>
>>>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> So, it seems that OidcClient and Auth0 is working but AAD and Google
>>>>>> are not.
>>>>>> While I've never tested AAD, I have tested Google so there may be a
>>>>>> regression there.
>>>>>>
>>>>>> I notice that there are specific clients for both AzureAd and Google
>>>>>> now - see: http://www.pac4j.org/docs/clients/openid-connect.html#2
>>>>>> -clients
>>>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>>>
>>>>>> They also have corresponding profile creators.
>>>>>>
>>>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>>>> AzureAdOidcClient ?
>>>>>>
>>>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> Trying with "www.local.com" makes no difference for me. The problem
>>>>>>> is that it is not retrieving the stored user profile correctly after it
>>>>>>> redirects back to the Pac4jDispatcherFilter:
>>>>>>>
>>>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession:
>>>>>>> true
>>>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>> pac4jUserProfiles = null
>>>>>>>
>>>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>>>
>>>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>>>
>>>>>>> and it worked! Perhaps we should change from storing the profile in
>>>>>>> a cookie to an attribute on the session instead?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>>>> originally requested resource.
>>>>>>>>
>>>>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>>>>> that authentication/federation of the enduser has been done, that the
>>>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>>>> mapped it to whatever the effective user should be.
>>>>>>>>
>>>>>>>> Therefore, the KnoxSSO service can then extract the
>>>>>>>> PrimaryPrincipal from the normalized Java Subject, create the JWT
>>>>>>>> representation of the authentication event and set-cookie.
>>>>>>>> It then redirects the user agent to the originally requested
>>>>>>>> resource where the browser should present the cookie.
>>>>>>>>
>>>>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>>>>> is referenced in:
>>>>>>>>>
>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>> h/app/js/knoxauth.js
>>>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>>>> h/app/redirecting.jsp
>>>>>>>>>
>>>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Colm -
>>>>>>>>>>
>>>>>>>>>> Thanks for trying to reproduce.
>>>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>>>
>>>>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>>>
>>>>>>>>>> Can you switch to a full domain name?
>>>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>>>> file.
>>>>>>>>>>
>>>>>>>>>> You will also need to make sure to set the whitelist for the
>>>>>>>>>> domain name.
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>>
>>>>>>>>>> --larry
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>>>> following:
>>>>>>>>>>>
>>>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>>>> /api/v1/websso
>>>>>>>>>>>
>>>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>>>
>>>>>>>>>>> Colm.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>>>> If it isn't being set, it could be that it isn't getting past
>>>>>>>>>>>> the pac4j federation provider to the KnoxSSO service itself because there
>>>>>>>>>>>> is a redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>>>> accepted by the browser.
>>>>>>>>>>>>
>>>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>>>
>>>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>>>> unchartered waters here.
>>>>>>>>>>>>
>>>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>>>
>>>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>>>> pac4j.session.pac4jRequestedUrl,
>>>>>>>>>>>>> pac4j.session.oidcStateAttribute,
>>>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth*
>>>>>>>>>>>>> also would have failed.
>>>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure
>>>>>>>>>>>>> AD.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <
>>>>>>>>>>>>> lmccay@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox
>>>>>>>>>>>>>> URLs - to webhdfs.
>>>>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>>>>> things:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set
>>>>>>>>>>>>>> in your browser
>>>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think that ip address should work for this case actually
>>>>>>>>>>>>>> but there are differences in browsers that might not let it.
>>>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can
>>>>>>>>>>>>>>> you turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects
>>>>>>>>>>>>>>>> to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and
>>>>>>>>>>>>>>>> finally lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200".
>>>>>>>>>>>>>>>> This leads to redirection!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>>>
>>>>>>>>>>> Talend Community Coder
>>>>>>>>>>> http://coders.talend.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hi Colm/Larry,
I am not quite sure how to upgrade KNOX. I have setup KNOX with the full
HDP 2.6 stack. So the bundled version is KNOX 0.12.
The upgrades supported in HDP are w.r.t corresponding stack version only I
guess. Since 2.6 is the highest version (for HDP stack) available, I dont
know if KNOX can be seperately upgraded to any version higher than 0.12.
Colm/Sandeep: Could you please detail the KnoxSessionStore change steps
little more? Where do I find the source code? Do I need to get the whole
source code from git, change and build it again?
These are the pac4j jars available in my stack now:
[root@localhost topologies]# locate pac4j
/usr/hdp/2.6.1.0-129/knox/dep/j2e-pac4j-1.2.2.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-cas-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-config-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-core-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-http-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-oauth-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/dep/*pac4j-oidc-1.8.9.jar*
/usr/hdp/2.6.1.0-129/knox/dep/pac4j-saml-1.8.9.jar
/usr/hdp/2.6.1.0-129/knox/lib/
*gateway-provider-security-pac4j-0.12.0.2.6.1.0-129.jar*
/usr/hdp/2.6.1.0-129/knox/templates/pac4j-knoxsso.xml
/var/lib/knox/data-2.6.1.0-129/security/keystores/knoxsso_pac4j-credentials.jceks
/var/lib/knox/data-2.6.1.0-129/security/keystores/knoxtoken_pac4j-credentials.jceks
Would it work if I replace the pac4j oidc and gateway provider jar with
latest version?
Regards
Nisha
On Tue, Feb 20, 2018 at 9:07 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> Hi all,
>
> @Nisha - I assumed you were on 1.0.0 as well. I tested Google successfully
> with 0.13.0, so maybe we are running into different issues. Could you try
> Azure with 0.13.0 and see if it works?
>
> @Larry, switching to use GoogleOidcClient doesn't make any difference. I
> just changed KnoxSessionStore manually and copied it over into my
> distribution :-)
>
> Colm.
>
> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org> wrote:
>
>> Oh, bummer.
>>
>> @Colm - can you share how you changed the KnoxSessionStore with the
>> J2ESessionStore?
>>
>>
>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello Larry,
>>>
>>> Well, this is what I started with in my implementation. I tested again.
>>> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
>>> be recongnised in pac4j client.
>>> Thats when I used "OidcClient" itself".
>>> Error:
>>>
>>> Caused by: org.pac4j.core.exception.TechnicalException: No client found
>>> for name: AzureAdClient
>>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>>> at org.pac4j.core.client.DefaultClientFinder.find(DefaultClient
>>> Finder.java:56)
>>>
>>> Regards
>>> Nisha
>>>
>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> So, it seems that OidcClient and Auth0 is working but AAD and Google
>>>> are not.
>>>> While I've never tested AAD, I have tested Google so there may be a
>>>> regression there.
>>>>
>>>> I notice that there are specific clients for both AzureAd and Google
>>>> now - see: http://www.pac4j.org/docs/clients/openid-connect.html#2
>>>> -clients
>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>
>>>> They also have corresponding profile creators.
>>>>
>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>> AzureAdOidcClient ?
>>>>
>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> Trying with "www.local.com" makes no difference for me. The problem
>>>>> is that it is not retrieving the stored user profile correctly after it
>>>>> redirects back to the Pac4jDispatcherFilter:
>>>>>
>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession:
>>>>> true
>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles
>>>>> = null
>>>>>
>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>
>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>
>>>>> and it worked! Perhaps we should change from storing the profile in a
>>>>> cookie to an attribute on the session instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>> originally requested resource.
>>>>>>
>>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>>> that authentication/federation of the enduser has been done, that the
>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>> mapped it to whatever the effective user should be.
>>>>>>
>>>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>>>> from the normalized Java Subject, create the JWT representation of the
>>>>>> authentication event and set-cookie.
>>>>>> It then redirects the user agent to the originally requested resource
>>>>>> where the browser should present the cookie.
>>>>>>
>>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>>> is referenced in:
>>>>>>>
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/js/knoxauth.js
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/redirecting.jsp
>>>>>>>
>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Colm -
>>>>>>>>
>>>>>>>> Thanks for trying to reproduce.
>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>
>>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>
>>>>>>>> Can you switch to a full domain name?
>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>> file.
>>>>>>>>
>>>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>>>> name.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> --larry
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>> following:
>>>>>>>>>
>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>> /api/v1/websso
>>>>>>>>>
>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>> accepted by the browser.
>>>>>>>>>>
>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>
>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>> unchartered waters here.
>>>>>>>>>>
>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>
>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>
>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>>> would have failed.
>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lmccay@apache.org
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs
>>>>>>>>>>>> - to webhdfs.
>>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>>> things:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>>> your browser
>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>
>>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>> anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>> Nisha
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
Thanks Colm, to add to the GoogleOidcClient discussion, I think it is not
supposed to be used directly, instead Pac4J needs to be told about it by
using the "oidc.type = google" property [1]
which unfortunately breaks everything :(
[1] http://www.pac4j.org/docs/config.html#2-the-pac4j-config-module
On Tue, Feb 20, 2018 at 10:37 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> Hi all,
>
> @Nisha - I assumed you were on 1.0.0 as well. I tested Google successfully
> with 0.13.0, so maybe we are running into different issues. Could you try
> Azure with 0.13.0 and see if it works?
>
> @Larry, switching to use GoogleOidcClient doesn't make any difference. I
> just changed KnoxSessionStore manually and copied it over into my
> distribution :-)
>
> Colm.
>
> On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org> wrote:
>
>> Oh, bummer.
>>
>> @Colm - can you share how you changed the KnoxSessionStore with the
>> J2ESessionStore?
>>
>>
>> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello Larry,
>>>
>>> Well, this is what I started with in my implementation. I tested again.
>>> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
>>> be recongnised in pac4j client.
>>> Thats when I used "OidcClient" itself".
>>> Error:
>>>
>>> Caused by: org.pac4j.core.exception.TechnicalException: No client found
>>> for name: AzureAdClient
>>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>>> at org.pac4j.core.client.DefaultClientFinder.find(DefaultClient
>>> Finder.java:56)
>>>
>>> Regards
>>> Nisha
>>>
>>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> So, it seems that OidcClient and Auth0 is working but AAD and Google
>>>> are not.
>>>> While I've never tested AAD, I have tested Google so there may be a
>>>> regression there.
>>>>
>>>> I notice that there are specific clients for both AzureAd and Google
>>>> now - see: http://www.pac4j.org/docs/clients/openid-connect.html#2
>>>> -clients
>>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>>
>>>> They also have corresponding profile creators.
>>>>
>>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>>> AzureAdOidcClient ?
>>>>
>>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> Trying with "www.local.com" makes no difference for me. The problem
>>>>> is that it is not retrieving the stored user profile correctly after it
>>>>> redirects back to the Pac4jDispatcherFilter:
>>>>>
>>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession:
>>>>> true
>>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles
>>>>> = null
>>>>>
>>>>> However, I replaced the KnoxSessionStore with the following:
>>>>>
>>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>>
>>>>> and it worked! Perhaps we should change from storing the profile in a
>>>>> cookie to an attribute on the session instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the
>>>>>> originally requested resource.
>>>>>>
>>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>>> that authentication/federation of the enduser has been done, that the
>>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>>> mapped it to whatever the effective user should be.
>>>>>>
>>>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>>>> from the normalized Java Subject, create the JWT representation of the
>>>>>> authentication event and set-cookie.
>>>>>> It then redirects the user agent to the originally requested resource
>>>>>> where the browser should present the cookie.
>>>>>>
>>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>>> is referenced in:
>>>>>>>
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/js/knoxauth.js
>>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>>> h/app/redirecting.jsp
>>>>>>>
>>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Colm -
>>>>>>>>
>>>>>>>> Thanks for trying to reproduce.
>>>>>>>> Starting to get concerned about a regression....
>>>>>>>>
>>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>>
>>>>>>>> Can you switch to a full domain name?
>>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts
>>>>>>>> file.
>>>>>>>>
>>>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>>>> name.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> --larry
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>>> coheigea@apache.org> wrote:
>>>>>>>>
>>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>>> following:
>>>>>>>>>
>>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>>> pac4jRequestedUrl = null
>>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>>> /api/v1/websso
>>>>>>>>>
>>>>>>>>> It is getting an access token correctly and trying to redirect
>>>>>>>>> back to the original URL. However from the logs above it seems to never hit
>>>>>>>>> the original URL but instead hits the "redirectUrl". Is some parsing
>>>>>>>>> supposed to take place to extract "originalUrl" from the
>>>>>>>>> "pac4jRequestedUrl" parameter and redirect to this instead?
>>>>>>>>>
>>>>>>>>> Colm.
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the
>>>>>>>>>> SSOCookieProvider.
>>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>>> accepted by the browser.
>>>>>>>>>>
>>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>>
>>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>>> unchartered waters here.
>>>>>>>>>>
>>>>>>>>>> @Jerome - any insights?
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Larry,
>>>>>>>>>>>
>>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>>
>>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>>> would have failed.
>>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lmccay@apache.org
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs
>>>>>>>>>>>> - to webhdfs.
>>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>>> things:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>>> your browser
>>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>>
>>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>>> anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, after the authentication at Azure login page, it
>>>>>>>>>>>>>> gets into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>>> response.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for
>>>>>>>>>>>>>> credentials.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>> Nisha
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Colm O hEigeartaigh
>>>>>>>>>
>>>>>>>>> Talend Community Coder
>>>>>>>>> http://coders.talend.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Colm O hEigeartaigh <co...@apache.org>.
Hi all,
@Nisha - I assumed you were on 1.0.0 as well. I tested Google successfully
with 0.13.0, so maybe we are running into different issues. Could you try
Azure with 0.13.0 and see if it works?
@Larry, switching to use GoogleOidcClient doesn't make any difference. I
just changed KnoxSessionStore manually and copied it over into my
distribution :-)
Colm.
On Tue, Feb 20, 2018 at 1:43 PM, larry mccay <lm...@apache.org> wrote:
> Oh, bummer.
>
> @Colm - can you share how you changed the KnoxSessionStore with the
> J2ESessionStore?
>
>
> On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello Larry,
>>
>> Well, this is what I started with in my implementation. I tested again.
>> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
>> be recongnised in pac4j client.
>> Thats when I used "OidcClient" itself".
>> Error:
>>
>> Caused by: org.pac4j.core.exception.TechnicalException: No client found
>> for name: AzureAdClient
>> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
>> at org.pac4j.core.client.DefaultClientFinder.find(DefaultClient
>> Finder.java:56)
>>
>> Regards
>> Nisha
>>
>> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> So, it seems that OidcClient and Auth0 is working but AAD and Google are
>>> not.
>>> While I've never tested AAD, I have tested Google so there may be a
>>> regression there.
>>>
>>> I notice that there are specific clients for both AzureAd and Google now
>>> - see: http://www.pac4j.org/docs/clients/openid-connect.html#2-clients
>>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>>
>>> They also have corresponding profile creators.
>>>
>>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>>> AzureAdOidcClient ?
>>>
>>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> Trying with "www.local.com" makes no difference for me. The problem is
>>>> that it is not retrieving the stored user profile correctly after it
>>>> redirects back to the Pac4jDispatcherFilter:
>>>>
>>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
>>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles
>>>> = null
>>>>
>>>> However, I replaced the KnoxSessionStore with the following:
>>>>
>>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>>
>>>> and it worked! Perhaps we should change from storing the profile in a
>>>> cookie to an attribute on the session instead?
>>>>
>>>> Colm.
>>>>
>>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>>>>> requested resource.
>>>>>
>>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>>> that authentication/federation of the enduser has been done, that the
>>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>>> mapped it to whatever the effective user should be.
>>>>>
>>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>>> from the normalized Java Subject, create the JWT representation of the
>>>>> authentication event and set-cookie.
>>>>> It then redirects the user agent to the originally requested resource
>>>>> where the browser should present the cookie.
>>>>>
>>>>> In this case, SSOCookieProvider will be then looking for the cookie
>>>>> and will redirect back to KnoxSSO if it is not present.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>>> coheigea@apache.org> wrote:
>>>>>
>>>>>> OK I'll check it. A question though - where is the redirection via
>>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>>> is referenced in:
>>>>>>
>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>> h/app/js/knoxauth.js
>>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>>> h/app/redirecting.jsp
>>>>>>
>>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>>
>>>>>> Colm.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Colm -
>>>>>>>
>>>>>>> Thanks for trying to reproduce.
>>>>>>> Starting to get concerned about a regression....
>>>>>>>
>>>>>>> I see that you are using localhost - that will definitely present
>>>>>>> problems for Set-Cookie with some browsers.
>>>>>>> I know that Chrome will not accept it for sure.
>>>>>>>
>>>>>>> Can you switch to a full domain name?
>>>>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>>>>
>>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>>> name.
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> --larry
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>>> coheigea@apache.org> wrote:
>>>>>>>
>>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>>> following:
>>>>>>>>
>>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>>> pac4jRequestedUrl = null
>>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137))
>>>>>>>> - redirectUrl: https://localhost:8443/gateway
>>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>>> /api/v1/websso
>>>>>>>>
>>>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>>>> parameter and redirect to this instead?
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>>> accepted by the browser.
>>>>>>>>>
>>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>>
>>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>>> unchartered waters here.
>>>>>>>>>
>>>>>>>>> @Jerome - any insights?
>>>>>>>>>
>>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Larry,
>>>>>>>>>>
>>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>>
>>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>>> would have failed.
>>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>>>> to webhdfs.
>>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>>> things:
>>>>>>>>>>>
>>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>>> your browser
>>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>>
>>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>>> anyway.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Sandeep
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>>>> response.
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to
>>>>>>>>>>>>> *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>>
>>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>>
>>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>>
>>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>>> appreciated.
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Nisha
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Colm O hEigeartaigh
>>>>>>>>
>>>>>>>> Talend Community Coder
>>>>>>>> http://coders.talend.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Colm O hEigeartaigh
>>>>>>
>>>>>> Talend Community Coder
>>>>>> http://coders.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
Oh, bummer.
@Colm - can you share how you changed the KnoxSessionStore with the
J2ESessionStore?
On Tue, Feb 20, 2018 at 8:39 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Hello Larry,
>
> Well, this is what I started with in my implementation. I tested again.
> Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
> be recongnised in pac4j client.
> Thats when I used "OidcClient" itself".
> Error:
>
> Caused by: org.pac4j.core.exception.TechnicalException: No client found
> for name: AzureAdClient
> at org.pac4j.core.client.Clients.findClient(Clients.java:147)
> at org.pac4j.core.client.DefaultClientFinder.find(
> DefaultClientFinder.java:56)
>
> Regards
> Nisha
>
> On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
>
>> So, it seems that OidcClient and Auth0 is working but AAD and Google are
>> not.
>> While I've never tested AAD, I have tested Google so there may be a
>> regression there.
>>
>> I notice that there are specific clients for both AzureAd and Google now
>> - see: http://www.pac4j.org/docs/clients/openid-connect.html#2-clients
>> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-oid
>> c/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
>> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-oi
>> dc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>>
>> They also have corresponding profile creators.
>>
>> Instead of the generic OidcClient, can we try GoogleOidcClient and
>> AzureAdOidcClient ?
>>
>> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> Trying with "www.local.com" makes no difference for me. The problem is
>>> that it is not retrieving the stored user profile correctly after it
>>> redirects back to the Pac4jDispatcherFilter:
>>>
>>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
>>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles =
>>> null
>>>
>>> However, I replaced the KnoxSessionStore with the following:
>>>
>>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/ma
>>> in/java/org/pac4j/core/context/session/J2ESessionStore.java
>>>
>>> and it worked! Perhaps we should change from storing the profile in a
>>> cookie to an attribute on the session instead?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>>>> requested resource.
>>>>
>>>> The KnoxSSO service assumes that by the time the request gets to it
>>>> that authentication/federation of the enduser has been done, that the
>>>> enduser is authorized to use knoxsso and that the identity assertion has
>>>> mapped it to whatever the effective user should be.
>>>>
>>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>>> from the normalized Java Subject, create the JWT representation of the
>>>> authentication event and set-cookie.
>>>> It then redirects the user agent to the originally requested resource
>>>> where the browser should present the cookie.
>>>>
>>>> In this case, SSOCookieProvider will be then looking for the cookie and
>>>> will redirect back to KnoxSSO if it is not present.
>>>>
>>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> OK I'll check it. A question though - where is the redirection via
>>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>>> is referenced in:
>>>>>
>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>> h/app/js/knoxauth.js
>>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>>> h/app/redirecting.jsp
>>>>>
>>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Colm -
>>>>>>
>>>>>> Thanks for trying to reproduce.
>>>>>> Starting to get concerned about a regression....
>>>>>>
>>>>>> I see that you are using localhost - that will definitely present
>>>>>> problems for Set-Cookie with some browsers.
>>>>>> I know that Chrome will not accept it for sure.
>>>>>>
>>>>>> Can you switch to a full domain name?
>>>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>>>
>>>>>> You will also need to make sure to set the whitelist for the domain
>>>>>> name.
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> --larry
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>>> coheigea@apache.org> wrote:
>>>>>>
>>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>>> following:
>>>>>>>
>>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>>> pac4jRequestedUrl = null
>>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>>>> redirectUrl: https://localhost:8443/gateway
>>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>>> /api/v1/websso
>>>>>>>
>>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>>> parameter and redirect to this instead?
>>>>>>>
>>>>>>> Colm.
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>>> accepted by the browser.
>>>>>>>>
>>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>>
>>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>>> unchartered waters here.
>>>>>>>>
>>>>>>>> @Jerome - any insights?
>>>>>>>>
>>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Larry,
>>>>>>>>>
>>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>>
>>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>>> would have failed.
>>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>>> to webhdfs.
>>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>>> things:
>>>>>>>>>>
>>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>>> your browser
>>>>>>>>>> 2. if not, you may want to set an actual domain in your
>>>>>>>>>> /etc/hosts or something that you can reference - I use
>>>>>>>>>> www.local.com to map to localhost
>>>>>>>>>>
>>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>>> anyway.
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Nisha,
>>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Sandeep
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>>
>>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>>> response.
>>>>>>>>>>>>
>>>>>>>>>>>> *Details:*
>>>>>>>>>>>>
>>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>>
>>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with
>>>>>>>>>>>> code, session and state variables passed as below:
>>>>>>>>>>>>
>>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>>
>>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>>
>>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>>
>>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1. <topology>
>>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>>> 3. <provider>
>>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>>> 8. </provider>
>>>>>>>>>>>> 9. <provider>
>>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>>> 13. <param>
>>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>>> 16. </param>
>>>>>>>>>>>> 17. <param>
>>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>>> 20. </param>
>>>>>>>>>>>> 21. <param>
>>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>>> 24. </param>
>>>>>>>>>>>> 25. <param>
>>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>>> 28. </param>
>>>>>>>>>>>> 29. <param>
>>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>>> 32. </param>
>>>>>>>>>>>> 33. </provider>
>>>>>>>>>>>> 34. <provider>
>>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>>> 38. </provider>
>>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>>> 40. <application>
>>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>>> 42. </application>
>>>>>>>>>>>> 43. <service>
>>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>>> 45. <param>
>>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>>> 48. </param>
>>>>>>>>>>>> 49. <param>
>>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>>> 52. </param>
>>>>>>>>>>>> 53. <param>
>>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>>> 56. </param>
>>>>>>>>>>>> 57. </service>
>>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>>
>>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>>
>>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>>
>>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>>
>>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>>> appreciated.
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Nisha
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>> Nisha
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Colm O hEigeartaigh
>>>>>>>
>>>>>>> Talend Community Coder
>>>>>>> http://coders.talend.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hello Larry,
Well, this is what I started with in my implementation. I tested again.
Both AzureADClient (AzureADOidcClient), GoogleOidcClient does not seem to
be recongnised in pac4j client.
Thats when I used "OidcClient" itself".
Error:
Caused by: org.pac4j.core.exception.TechnicalException: No client found for
name: AzureAdClient
at org.pac4j.core.client.Clients.findClient(Clients.java:147)
at
org.pac4j.core.client.DefaultClientFinder.find(DefaultClientFinder.java:56)
Regards
Nisha
On Tue, Feb 20, 2018 at 6:59 PM, larry mccay <lm...@apache.org> wrote:
> So, it seems that OidcClient and Auth0 is working but AAD and Google are
> not.
> While I've never tested AAD, I have tested Google so there may be a
> regression there.
>
> I notice that there are specific clients for both AzureAd and Google now -
> see: http://www.pac4j.org/docs/clients/openid-connect.html#2-clients
> Google: https://github.com/pac4j/pac4j/blob/master/pac4j-
> oidc/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
> AzureAd: https://github.com/pac4j/pac4j/blob/master/pac4j-
> oidc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
>
> They also have corresponding profile creators.
>
> Instead of the generic OidcClient, can we try GoogleOidcClient and
> AzureAdOidcClient ?
>
> On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
>> Trying with "www.local.com" makes no difference for me. The problem is
>> that it is not retrieving the stored user profile correctly after it
>> redirects back to the Pac4jDispatcherFilter:
>>
>> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
>> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
>> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles =
>> null
>>
>> However, I replaced the KnoxSessionStore with the following:
>>
>> https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/
>> main/java/org/pac4j/core/context/session/J2ESessionStore.java
>>
>> and it worked! Perhaps we should change from storing the profile in a
>> cookie to an attribute on the session instead?
>>
>> Colm.
>>
>> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>>> requested resource.
>>>
>>> The KnoxSSO service assumes that by the time the request gets to it that
>>> authentication/federation of the enduser has been done, that the enduser is
>>> authorized to use knoxsso and that the identity assertion has mapped it to
>>> whatever the effective user should be.
>>>
>>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal
>>> from the normalized Java Subject, create the JWT representation of the
>>> authentication event and set-cookie.
>>> It then redirects the user agent to the originally requested resource
>>> where the browser should present the cookie.
>>>
>>> In this case, SSOCookieProvider will be then looking for the cookie and
>>> will redirect back to KnoxSSO if it is not present.
>>>
>>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> OK I'll check it. A question though - where is the redirection via
>>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>>> is referenced in:
>>>>
>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>> h/app/js/knoxauth.js
>>>> gateway-applications/src/main/resources/applications/knoxaut
>>>> h/app/redirecting.jsp
>>>>
>>>> However, I'm not using the knoxauth application (with pac4j)?
>>>>
>>>> Colm.
>>>>
>>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> Hi Colm -
>>>>>
>>>>> Thanks for trying to reproduce.
>>>>> Starting to get concerned about a regression....
>>>>>
>>>>> I see that you are using localhost - that will definitely present
>>>>> problems for Set-Cookie with some browsers.
>>>>> I know that Chrome will not accept it for sure.
>>>>>
>>>>> Can you switch to a full domain name?
>>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>>
>>>>> You will also need to make sure to set the whitelist for the domain
>>>>> name.
>>>>>
>>>>> thanks,
>>>>>
>>>>> --larry
>>>>>
>>>>>
>>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>>> coheigea@apache.org> wrote:
>>>>>
>>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>>> following:
>>>>>>
>>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>>> (KnoxSessionStore.java:get(91)) - Get from session:
>>>>>> pac4jRequestedUrl = https://localhost:8443/gateway
>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>>> (KnoxSessionStore.java:set(107)) - Save in session:
>>>>>> pac4jRequestedUrl = null
>>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>>> redirectUrl: https://localhost:8443/gateway
>>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>>> /api/v1/websso
>>>>>>
>>>>>> It is getting an access token correctly and trying to redirect back
>>>>>> to the original URL. However from the logs above it seems to never hit the
>>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>>> parameter and redirect to this instead?
>>>>>>
>>>>>> Colm.
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>>> accepted by the browser.
>>>>>>>
>>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>>
>>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>>> unchartered waters here.
>>>>>>>
>>>>>>> @Jerome - any insights?
>>>>>>>
>>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <
>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello Larry,
>>>>>>>>
>>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>>
>>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>>> would have failed.
>>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> It looks like you may be using ip addresses for your Knox URLs -
>>>>>>>>> to webhdfs.
>>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>>> things:
>>>>>>>>>
>>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in
>>>>>>>>> your browser
>>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>>> to localhost
>>>>>>>>>
>>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>>> Also, if you had another service on another ip address, the
>>>>>>>>> browser would not present the cookie - so this is good to be aware of
>>>>>>>>> anyway.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Nisha,
>>>>>>>>>> Can you share details of "mycluster" topology ? also, can you
>>>>>>>>>> turn up the logs to debug and share them along with the audit log that
>>>>>>>>>> would help us to understand the problem better.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Sandeep
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>>
>>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>>> response.
>>>>>>>>>>>
>>>>>>>>>>> *Details:*
>>>>>>>>>>>
>>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>>
>>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>>
>>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>>
>>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>>
>>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>>
>>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>>
>>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>>
>>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 1. <topology>
>>>>>>>>>>> 2. <gateway>
>>>>>>>>>>> 3. <provider>
>>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>>> 8. </provider>
>>>>>>>>>>> 9. <provider>
>>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>>> 13. <param>
>>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>>> 16. </param>
>>>>>>>>>>> 17. <param>
>>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>>> 20. </param>
>>>>>>>>>>> 21. <param>
>>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>>> 24. </param>
>>>>>>>>>>> 25. <param>
>>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>>> 28. </param>
>>>>>>>>>>> 29. <param>
>>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>>> 32. </param>
>>>>>>>>>>> 33. </provider>
>>>>>>>>>>> 34. <provider>
>>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>>> 38. </provider>
>>>>>>>>>>> 39. </gateway>
>>>>>>>>>>> 40. <application>
>>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>>> 42. </application>
>>>>>>>>>>> 43. <service>
>>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>>> 45. <param>
>>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>>> 48. </param>
>>>>>>>>>>> 49. <param>
>>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>>> 52. </param>
>>>>>>>>>>> 53. <param>
>>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>>> 56. </param>
>>>>>>>>>>> 57. </service>
>>>>>>>>>>> 58. </topology>
>>>>>>>>>>>
>>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>>
>>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>>
>>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>>> leads to redirection!
>>>>>>>>>>>
>>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>>> appreciated.
>>>>>>>>>>> Regards
>>>>>>>>>>> Nisha
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>> Nisha
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Colm O hEigeartaigh
>>>>>>
>>>>>> Talend Community Coder
>>>>>> http://coders.talend.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
So, it seems that OidcClient and Auth0 is working but AAD and Google are
not.
While I've never tested AAD, I have tested Google so there may be a
regression there.
I notice that there are specific clients for both AzureAd and Google now -
see: http://www.pac4j.org/docs/clients/openid-connect.html#2-clients
Google:
https://github.com/pac4j/pac4j/blob/master/pac4j-oidc/src/main/java/org/pac4j/oidc/client/GoogleOidcClient.java
AzureAd:
https://github.com/pac4j/pac4j/blob/master/pac4j-oidc/src/main/java/org/pac4j/oidc/client/AzureAdClient.java
They also have corresponding profile creators.
Instead of the generic OidcClient, can we try GoogleOidcClient and
AzureAdOidcClient ?
On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> Trying with "www.local.com" makes no difference for me. The problem is
> that it is not retrieving the stored user profile correctly after it
> redirects back to the Pac4jDispatcherFilter:
>
> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles =
> null
>
> However, I replaced the KnoxSessionStore with the following:
>
> https://github.com/pac4j/pac4j/blob/master/pac4j-core/
> src/main/java/org/pac4j/core/context/session/J2ESessionStore.java
>
> and it worked! Perhaps we should change from storing the profile in a
> cookie to an attribute on the session instead?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>
>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>> requested resource.
>>
>> The KnoxSSO service assumes that by the time the request gets to it that
>> authentication/federation of the enduser has been done, that the enduser is
>> authorized to use knoxsso and that the identity assertion has mapped it to
>> whatever the effective user should be.
>>
>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal from
>> the normalized Java Subject, create the JWT representation of the
>> authentication event and set-cookie.
>> It then redirects the user agent to the originally requested resource
>> where the browser should present the cookie.
>>
>> In this case, SSOCookieProvider will be then looking for the cookie and
>> will redirect back to KnoxSSO if it is not present.
>>
>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> OK I'll check it. A question though - where is the redirection via
>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>> is referenced in:
>>>
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/js/knoxauth.js
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/redirecting.jsp
>>>
>>> However, I'm not using the knoxauth application (with pac4j)?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> Hi Colm -
>>>>
>>>> Thanks for trying to reproduce.
>>>> Starting to get concerned about a regression....
>>>>
>>>> I see that you are using localhost - that will definitely present
>>>> problems for Set-Cookie with some browsers.
>>>> I know that Chrome will not accept it for sure.
>>>>
>>>> Can you switch to a full domain name?
>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>
>>>> You will also need to make sure to set the whitelist for the domain
>>>> name.
>>>>
>>>> thanks,
>>>>
>>>> --larry
>>>>
>>>>
>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>> following:
>>>>>
>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>>> = null
>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>> redirectUrl: https://localhost:8443/gateway
>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>> /api/v1/websso
>>>>>
>>>>> It is getting an access token correctly and trying to redirect back to
>>>>> the original URL. However from the logs above it seems to never hit the
>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>> parameter and redirect to this instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>> accepted by the browser.
>>>>>>
>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>
>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>> unchartered waters here.
>>>>>>
>>>>>> @Jerome - any insights?
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>
>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>> would have failed.
>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>>> webhdfs.
>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>> things:
>>>>>>>>
>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>>> browser
>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>> to localhost
>>>>>>>>
>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Nisha,
>>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>>> help us to understand the problem better.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Sandeep
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>
>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>> response.
>>>>>>>>>>
>>>>>>>>>> *Details:*
>>>>>>>>>>
>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>
>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>
>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>
>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>
>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>
>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>
>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>
>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. <topology>
>>>>>>>>>> 2. <gateway>
>>>>>>>>>> 3. <provider>
>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>> 8. </provider>
>>>>>>>>>> 9. <provider>
>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>> 13. <param>
>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>> 16. </param>
>>>>>>>>>> 17. <param>
>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>> 20. </param>
>>>>>>>>>> 21. <param>
>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>> 24. </param>
>>>>>>>>>> 25. <param>
>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>> 28. </param>
>>>>>>>>>> 29. <param>
>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>> 32. </param>
>>>>>>>>>> 33. </provider>
>>>>>>>>>> 34. <provider>
>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>> 38. </provider>
>>>>>>>>>> 39. </gateway>
>>>>>>>>>> 40. <application>
>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>> 42. </application>
>>>>>>>>>> 43. <service>
>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>> 45. <param>
>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>> 48. </param>
>>>>>>>>>> 49. <param>
>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>> 52. </param>
>>>>>>>>>> 53. <param>
>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>> 56. </param>
>>>>>>>>>> 57. </service>
>>>>>>>>>> 58. </topology>
>>>>>>>>>>
>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>
>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>
>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>> leads to redirection!
>>>>>>>>>>
>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>> appreciated.
>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hi Colm,
Hmm, even I enabled debug logs and saw something similar for my issue with
AzureAD connect, where the pac4jUserProfile = null.
However, I tested the whole connection with OidcClient and Auth.com as the
server for authentication, and I get appropriate pac4jUserProfile filled.
See below:
2018-02-20 13:04:12,267 DEBUG hadoop.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1/user
2018-02-20 13:04:12,300 DEBUG federation.jwt
(SSOCookieFederationFilter.java:doFilter(109)) - Sending redirect to:
https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?originalUrl=https://x.x.2.3:8442/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS
2018-02-20 13:04:12,567 DEBUG hadoop.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso
2018-02-20 13:04:12,681 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(90)) - Get from session: pac4jUserProfile = null
2018-02-20 13:04:12,683 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(105)) - Save in session: pac4jRequestedUrl =
https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?originalUrl=https://x.x.2.3:8442/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS
2018-02-20 13:04:12,879 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(90)) - Get from session:
OidcClient$attemptedAuthentication = null
2018-02-20 13:04:13,575 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(105)) - Save in session: oidcStateAttribute =
OYOByx5w0VUodX6eSa4nZGcU4X7Z-DyMOzk8_Bc1WPk
2018-02-20 13:04:40,914 DEBUG hadoop.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso
2018-02-20 13:04:41,097 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(90)) - Get from session: oidcStateAttribute =
OYOByx5w0VUodX6eSa4nZGcU4X7Z-DyMOzk8_Bc1WPk
2018-02-20 13:04:41,099 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(105)) - Save in session:
OidcClient$attemptedAuthentication =
2018-02-20 13:04:41,439 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(105)) - Save in session: *pac4jUserProfile =
<OidcProfile> | id: auth0|5a8bbf0eb4024d4087f89741 | attributes:
{sub=auth0|5a8bbf0eb4024d4087f89741, email_verified=true,
updated_at=2018-02-20T13:04:40.651Z, nickname=nisha.menon,
name=nisha.menon@xyz.com <ni...@xyz.com>,
picture=https://s.gravatar.com/avatar/9d88657bce30a56f89b89e7da5348339?s=480&r=pg&d=https%3A%2F%2Fcdn.auth0.com%2Favatars%2Fni.png
<https://s.gravatar.com/avatar/9d88657bce30a56f89b89e7da5348339?s=480&r=pg&d=https%3A%2F%2Fcdn.auth0.com%2Favatars%2Fni.png>,
email=nisha.menon@xyz.com <ni...@xyz.com>} | roles: [] | permissions:
[] | isRemembered: false |*
2018-02-20 13:04:41,620 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(90)) - Get from session: pac4jRequestedUrl =
https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?originalUrl=https://x.x.2.3:8442/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS
Why is the behaviour different in different cases?
Regards
Nisha
On Tue, Feb 20, 2018 at 4:44 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> Trying with "www.local.com" makes no difference for me. The problem is
> that it is not retrieving the stored user profile correctly after it
> redirects back to the Pac4jDispatcherFilter:
>
> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles =
> null
>
> However, I replaced the KnoxSessionStore with the following:
>
> https://github.com/pac4j/pac4j/blob/master/pac4j-core/
> src/main/java/org/pac4j/core/context/session/J2ESessionStore.java
>
> and it worked! Perhaps we should change from storing the profile in a
> cookie to an attribute on the session instead?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>
>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>> requested resource.
>>
>> The KnoxSSO service assumes that by the time the request gets to it that
>> authentication/federation of the enduser has been done, that the enduser is
>> authorized to use knoxsso and that the identity assertion has mapped it to
>> whatever the effective user should be.
>>
>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal from
>> the normalized Java Subject, create the JWT representation of the
>> authentication event and set-cookie.
>> It then redirects the user agent to the originally requested resource
>> where the browser should present the cookie.
>>
>> In this case, SSOCookieProvider will be then looking for the cookie and
>> will redirect back to KnoxSSO if it is not present.
>>
>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> OK I'll check it. A question though - where is the redirection via
>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>> is referenced in:
>>>
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/js/knoxauth.js
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/redirecting.jsp
>>>
>>> However, I'm not using the knoxauth application (with pac4j)?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> Hi Colm -
>>>>
>>>> Thanks for trying to reproduce.
>>>> Starting to get concerned about a regression....
>>>>
>>>> I see that you are using localhost - that will definitely present
>>>> problems for Set-Cookie with some browsers.
>>>> I know that Chrome will not accept it for sure.
>>>>
>>>> Can you switch to a full domain name?
>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>
>>>> You will also need to make sure to set the whitelist for the domain
>>>> name.
>>>>
>>>> thanks,
>>>>
>>>> --larry
>>>>
>>>>
>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>> following:
>>>>>
>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>>> = null
>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>> redirectUrl: https://localhost:8443/gateway
>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>> /api/v1/websso
>>>>>
>>>>> It is getting an access token correctly and trying to redirect back to
>>>>> the original URL. However from the logs above it seems to never hit the
>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>> parameter and redirect to this instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>> accepted by the browser.
>>>>>>
>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>
>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>> unchartered waters here.
>>>>>>
>>>>>> @Jerome - any insights?
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>
>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>> would have failed.
>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>>> webhdfs.
>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>> things:
>>>>>>>>
>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>>> browser
>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>> to localhost
>>>>>>>>
>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Nisha,
>>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>>> help us to understand the problem better.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Sandeep
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>
>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>> response.
>>>>>>>>>>
>>>>>>>>>> *Details:*
>>>>>>>>>>
>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>
>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>
>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>
>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>
>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>
>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>
>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>
>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. <topology>
>>>>>>>>>> 2. <gateway>
>>>>>>>>>> 3. <provider>
>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>> 8. </provider>
>>>>>>>>>> 9. <provider>
>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>> 13. <param>
>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>> 16. </param>
>>>>>>>>>> 17. <param>
>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>> 20. </param>
>>>>>>>>>> 21. <param>
>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>> 24. </param>
>>>>>>>>>> 25. <param>
>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>> 28. </param>
>>>>>>>>>> 29. <param>
>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>> 32. </param>
>>>>>>>>>> 33. </provider>
>>>>>>>>>> 34. <provider>
>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>> 38. </provider>
>>>>>>>>>> 39. </gateway>
>>>>>>>>>> 40. <application>
>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>> 42. </application>
>>>>>>>>>> 43. <service>
>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>> 45. <param>
>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>> 48. </param>
>>>>>>>>>> 49. <param>
>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>> 52. </param>
>>>>>>>>>> 53. <param>
>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>> 56. </param>
>>>>>>>>>> 57. </service>
>>>>>>>>>> 58. </topology>
>>>>>>>>>>
>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>
>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>
>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>> leads to redirection!
>>>>>>>>>>
>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>> appreciated.
>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
I can confirm this, although there is not easy way to swap KnoxSessionStore
for J2ESessionStore, I had to update the source code.
For my case, I added a filter param "pac4j.session.store", I think this
would be useful !
On Tue, Feb 20, 2018 at 6:14 AM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> Trying with "www.local.com" makes no difference for me. The problem is
> that it is not retrieving the stored user profile correctly after it
> redirects back to the Pac4jDispatcherFilter:
>
> 2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
> (DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
> 2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles =
> null
>
> However, I replaced the KnoxSessionStore with the following:
>
> https://github.com/pac4j/pac4j/blob/master/pac4j-core/
> src/main/java/org/pac4j/core/context/session/J2ESessionStore.java
>
> and it worked! Perhaps we should change from storing the profile in a
> cookie to an attribute on the session instead?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
>
>> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
>> requested resource.
>>
>> The KnoxSSO service assumes that by the time the request gets to it that
>> authentication/federation of the enduser has been done, that the enduser is
>> authorized to use knoxsso and that the identity assertion has mapped it to
>> whatever the effective user should be.
>>
>> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal from
>> the normalized Java Subject, create the JWT representation of the
>> authentication event and set-cookie.
>> It then redirects the user agent to the originally requested resource
>> where the browser should present the cookie.
>>
>> In this case, SSOCookieProvider will be then looking for the cookie and
>> will redirect back to KnoxSSO if it is not present.
>>
>> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <coheigea@apache.org
>> > wrote:
>>
>>> OK I'll check it. A question though - where is the redirection via
>>> originalUrl supposed to take place? From the source I can see "originalUrl"
>>> is referenced in:
>>>
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/js/knoxauth.js
>>> gateway-applications/src/main/resources/applications/knoxaut
>>> h/app/redirecting.jsp
>>>
>>> However, I'm not using the knoxauth application (with pac4j)?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> Hi Colm -
>>>>
>>>> Thanks for trying to reproduce.
>>>> Starting to get concerned about a regression....
>>>>
>>>> I see that you are using localhost - that will definitely present
>>>> problems for Set-Cookie with some browsers.
>>>> I know that Chrome will not accept it for sure.
>>>>
>>>> Can you switch to a full domain name?
>>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>>
>>>> You will also need to make sure to set the whitelist for the domain
>>>> name.
>>>>
>>>> thanks,
>>>>
>>>> --larry
>>>>
>>>>
>>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>>> coheigea@apache.org> wrote:
>>>>
>>>>> I can reproduce the issue with Google. From the logs I see the
>>>>> following:
>>>>>
>>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>>> = null
>>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>>> redirectUrl: https://localhost:8443/gateway
>>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>>> /api/v1/websso
>>>>>
>>>>> It is getting an access token correctly and trying to redirect back to
>>>>> the original URL. However from the logs above it seems to never hit the
>>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>>> parameter and redirect to this instead?
>>>>>
>>>>> Colm.
>>>>>
>>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>>> accepted by the browser.
>>>>>>
>>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>>
>>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>>> unchartered waters here.
>>>>>>
>>>>>> @Jerome - any insights?
>>>>>>
>>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello Larry,
>>>>>>>
>>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>>
>>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>>> would have failed.
>>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>>> webhdfs.
>>>>>>>> In order to rule out cookie related issue can you do a couple
>>>>>>>> things:
>>>>>>>>
>>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>>> browser
>>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>>> or something that you can reference - I use www.local.com to map
>>>>>>>> to localhost
>>>>>>>>
>>>>>>>> I think that ip address should work for this case actually but
>>>>>>>> there are differences in browsers that might not let it.
>>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <
>>>>>>>> moresandeep@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Nisha,
>>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>>> help us to understand the problem better.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Sandeep
>>>>>>>>>
>>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>>
>>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>>> response.
>>>>>>>>>>
>>>>>>>>>> *Details:*
>>>>>>>>>>
>>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>>
>>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>>
>>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>>> session and state variables passed as below:
>>>>>>>>>>
>>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>>
>>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>>
>>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>>
>>>>>>>>>> After this, step 1 and 2 alternate several times and finally
>>>>>>>>>> lands up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>>
>>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. <topology>
>>>>>>>>>> 2. <gateway>
>>>>>>>>>> 3. <provider>
>>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>>> 8. </provider>
>>>>>>>>>> 9. <provider>
>>>>>>>>>> 10. <role>federation</role>
>>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>>> 13. <param>
>>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>>> 16. </param>
>>>>>>>>>> 17. <param>
>>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>>> 20. </param>
>>>>>>>>>> 21. <param>
>>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>>> 24. </param>
>>>>>>>>>> 25. <param>
>>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>>> 28. </param>
>>>>>>>>>> 29. <param>
>>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>>> 32. </param>
>>>>>>>>>> 33. </provider>
>>>>>>>>>> 34. <provider>
>>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>>> 36. <name>Default</name>
>>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>>> 38. </provider>
>>>>>>>>>> 39. </gateway>
>>>>>>>>>> 40. <application>
>>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>>> 42. </application>
>>>>>>>>>> 43. <service>
>>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>>> 45. <param>
>>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>>> 47. <value>false</value>
>>>>>>>>>> 48. </param>
>>>>>>>>>> 49. <param>
>>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>>> 51. <value>30000</value>
>>>>>>>>>> 52. </param>
>>>>>>>>>> 53. <param>
>>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>>> 56. </param>
>>>>>>>>>> 57. </service>
>>>>>>>>>> 58. </topology>
>>>>>>>>>>
>>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>>
>>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>>
>>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>>> leads to redirection!
>>>>>>>>>>
>>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>>> appreciated.
>>>>>>>>>> Regards
>>>>>>>>>> Nisha
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Colm O hEigeartaigh
>>>>>
>>>>> Talend Community Coder
>>>>> http://coders.talend.com
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Colm O hEigeartaigh <co...@apache.org>.
Trying with "www.local.com" makes no difference for me. The problem is that
it is not retrieving the stored user profile correctly after it redirects
back to the Pac4jDispatcherFilter:
2018-02-20 10:55:07,486 DEBUG engine.DefaultSecurityLogic
(DefaultSecurityLogic.java:perform(98)) - loadProfilesFromSession: true
2018-02-20 10:55:07,487 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(91)) - Get from session: pac4jUserProfiles = null
However, I replaced the KnoxSessionStore with the following:
https://github.com/pac4j/pac4j/blob/master/pac4j-core/src/main/java/org/pac4j/core/context/session/J2ESessionStore.java
and it worked! Perhaps we should change from storing the profile in a
cookie to an attribute on the session instead?
Colm.
On Mon, Feb 19, 2018 at 6:43 PM, larry mccay <lm...@apache.org> wrote:
> KnoxSSO service (WebSSOResource) uses it to redirect to the originally
> requested resource.
>
> The KnoxSSO service assumes that by the time the request gets to it that
> authentication/federation of the enduser has been done, that the enduser is
> authorized to use knoxsso and that the identity assertion has mapped it to
> whatever the effective user should be.
>
> Therefore, the KnoxSSO service can then extract the PrimaryPrincipal from
> the normalized Java Subject, create the JWT representation of the
> authentication event and set-cookie.
> It then redirects the user agent to the originally requested resource
> where the browser should present the cookie.
>
> In this case, SSOCookieProvider will be then looking for the cookie and
> will redirect back to KnoxSSO if it is not present.
>
> On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <co...@apache.org>
> wrote:
>
>> OK I'll check it. A question though - where is the redirection via
>> originalUrl supposed to take place? From the source I can see "originalUrl"
>> is referenced in:
>>
>> gateway-applications/src/main/resources/applications/knoxaut
>> h/app/js/knoxauth.js
>> gateway-applications/src/main/resources/applications/knoxaut
>> h/app/redirecting.jsp
>>
>> However, I'm not using the knoxauth application (with pac4j)?
>>
>> Colm.
>>
>> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> Hi Colm -
>>>
>>> Thanks for trying to reproduce.
>>> Starting to get concerned about a regression....
>>>
>>> I see that you are using localhost - that will definitely present
>>> problems for Set-Cookie with some browsers.
>>> I know that Chrome will not accept it for sure.
>>>
>>> Can you switch to a full domain name?
>>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>>
>>> You will also need to make sure to set the whitelist for the domain name.
>>>
>>> thanks,
>>>
>>> --larry
>>>
>>>
>>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>>> coheigea@apache.org> wrote:
>>>
>>>> I can reproduce the issue with Google. From the logs I see the
>>>> following:
>>>>
>>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl
>>>> = https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl
>>>> = null
>>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>>> redirectUrl: https://localhost:8443/gateway
>>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway
>>>> (GatewayFilter.java:doFilter(119)) - Received request: GET
>>>> /api/v1/websso
>>>>
>>>> It is getting an access token correctly and trying to redirect back to
>>>> the original URL. However from the logs above it seems to never hit the
>>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>>> parameter and redirect to this instead?
>>>>
>>>> Colm.
>>>>
>>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>>> If it isn't being set, it could be that it isn't getting past the
>>>>> pac4j federation provider to the KnoxSSO service itself because there is a
>>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>>> accepted by the browser.
>>>>>
>>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>>
>>>>> I haven't seen any examples of it working AAD - so we are in
>>>>> unchartered waters here.
>>>>>
>>>>> @Jerome - any insights?
>>>>>
>>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Larry,
>>>>>>
>>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>>
>>>>>> IP address and hostnames are mapped, else the *basic auth* also
>>>>>> would have failed.
>>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>>> webhdfs.
>>>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>>>
>>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>>> browser
>>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts
>>>>>>> or something that you can reference - I use www.local.com to map to
>>>>>>> localhost
>>>>>>>
>>>>>>> I think that ip address should work for this case actually but there
>>>>>>> are differences in browsers that might not let it.
>>>>>>> Also, if you had another service on another ip address, the browser
>>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <moresandeep@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hello Nisha,
>>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>>> help us to understand the problem better.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Sandeep
>>>>>>>>
>>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>>
>>>>>>>>> However, after the authentication at Azure login page, it gets
>>>>>>>>> into an infinite loop and does not give back the original REST call
>>>>>>>>> response.
>>>>>>>>>
>>>>>>>>> *Details:*
>>>>>>>>>
>>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>>
>>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>>
>>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>>> session and state variables passed as below:
>>>>>>>>>
>>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>>
>>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>>
>>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>>
>>>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>>
>>>>>>>>> This is my knoxsso.xml:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. <topology>
>>>>>>>>> 2. <gateway>
>>>>>>>>> 3. <provider>
>>>>>>>>> 4. <role>webappsec</role>
>>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>>> 8. </provider>
>>>>>>>>> 9. <provider>
>>>>>>>>> 10. <role>federation</role>
>>>>>>>>> 11. <name>pac4j</name>
>>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>>> 13. <param>
>>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>>> 16. </param>
>>>>>>>>> 17. <param>
>>>>>>>>> 18. <name>clientName</name>
>>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>>> 20. </param>
>>>>>>>>> 21. <param>
>>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>>> 24. </param>
>>>>>>>>> 25. <param>
>>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>>> 28. </param>
>>>>>>>>> 29. <param>
>>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>>> 32. </param>
>>>>>>>>> 33. </provider>
>>>>>>>>> 34. <provider>
>>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>>> 36. <name>Default</name>
>>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>>> 38. </provider>
>>>>>>>>> 39. </gateway>
>>>>>>>>> 40. <application>
>>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>>> 42. </application>
>>>>>>>>> 43. <service>
>>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>>> 45. <param>
>>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>>> 47. <value>false</value>
>>>>>>>>> 48. </param>
>>>>>>>>> 49. <param>
>>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>>> 51. <value>30000</value>
>>>>>>>>> 52. </param>
>>>>>>>>> 53. <param>
>>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>>> 56. </param>
>>>>>>>>> 57. </service>
>>>>>>>>> 58. </topology>
>>>>>>>>>
>>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>>
>>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>>
>>>>>>>>> It clearly shows Response status as "302" and not "200". This
>>>>>>>>> leads to redirection!
>>>>>>>>>
>>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>>> appreciated.
>>>>>>>>> Regards
>>>>>>>>> Nisha
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Colm O hEigeartaigh
>>>>
>>>> Talend Community Coder
>>>> http://coders.talend.com
>>>>
>>>
>>>
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
KnoxSSO service (WebSSOResource) uses it to redirect to the originally
requested resource.
The KnoxSSO service assumes that by the time the request gets to it that
authentication/federation of the enduser has been done, that the enduser is
authorized to use knoxsso and that the identity assertion has mapped it to
whatever the effective user should be.
Therefore, the KnoxSSO service can then extract the PrimaryPrincipal from
the normalized Java Subject, create the JWT representation of the
authentication event and set-cookie.
It then redirects the user agent to the originally requested resource where
the browser should present the cookie.
In this case, SSOCookieProvider will be then looking for the cookie and
will redirect back to KnoxSSO if it is not present.
On Mon, Feb 19, 2018 at 1:15 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> OK I'll check it. A question though - where is the redirection via
> originalUrl supposed to take place? From the source I can see "originalUrl"
> is referenced in:
>
> gateway-applications/src/main/resources/applications/
> knoxauth/app/js/knoxauth.js
> gateway-applications/src/main/resources/applications/
> knoxauth/app/redirecting.jsp
>
> However, I'm not using the knoxauth application (with pac4j)?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
>
>> Hi Colm -
>>
>> Thanks for trying to reproduce.
>> Starting to get concerned about a regression....
>>
>> I see that you are using localhost - that will definitely present
>> problems for Set-Cookie with some browsers.
>> I know that Chrome will not accept it for sure.
>>
>> Can you switch to a full domain name?
>> I usually just map localhost to www.local.com in my /etc/hosts file.
>>
>> You will also need to make sure to set the whitelist for the domain name.
>>
>> thanks,
>>
>> --larry
>>
>>
>> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <
>> coheigea@apache.org> wrote:
>>
>>> I can reproduce the issue with Google. From the logs I see the following:
>>>
>>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
>>> https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?or
>>> iginalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/we
>>> bhdfs/v1/data/LICENSE.txt?op=OPEN
>>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
>>> null
>>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>>> redirectUrl: https://localhost:8443/gateway
>>> /knoxssopac4j/api/v1/websso?originalUrl=https://localhost:84
>>> 43/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
>>> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
>>> - Received request: GET /api/v1/websso
>>>
>>> It is getting an access token correctly and trying to redirect back to
>>> the original URL. However from the logs above it seems to never hit the
>>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>>> parameter and redirect to this instead?
>>>
>>> Colm.
>>>
>>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>>> If it isn't being set, it could be that it isn't getting past the pac4j
>>>> federation provider to the KnoxSSO service itself because there is a
>>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>>> accepted by the browser.
>>>>
>>>> The audit log does seem to be getting a redirect from pac4j.
>>>>
>>>> I haven't seen any examples of it working AAD - so we are in
>>>> unchartered waters here.
>>>>
>>>> @Jerome - any insights?
>>>>
>>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Larry,
>>>>>
>>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>>> In SSO provider with pac4j, I can see cookies like:
>>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>>> pac4j.session.oidcNonceAttribute etc.
>>>>>
>>>>> IP address and hostnames are mapped, else the *basic auth* also would
>>>>> have failed.
>>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>>
>>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>>> webhdfs.
>>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>>
>>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>>> browser
>>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>>>> something that you can reference - I use www.local.com to map to
>>>>>> localhost
>>>>>>
>>>>>> I think that ip address should work for this case actually but there
>>>>>> are differences in browsers that might not let it.
>>>>>> Also, if you had another service on another ip address, the browser
>>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Nisha,
>>>>>>> Can you share details of "mycluster" topology ? also, can you turn
>>>>>>> up the logs to debug and share them along with the audit log that would
>>>>>>> help us to understand the problem better.
>>>>>>>
>>>>>>> Best,
>>>>>>> Sandeep
>>>>>>>
>>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <
>>>>>>> nisha.menon16@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>>
>>>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>>>
>>>>>>>> *Details:*
>>>>>>>>
>>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>>
>>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>>
>>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>>> session and state variables passed as below:
>>>>>>>>
>>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>>
>>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>>
>>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>>
>>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>>
>>>>>>>> This is my knoxsso.xml:
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. <topology>
>>>>>>>> 2. <gateway>
>>>>>>>> 3. <provider>
>>>>>>>> 4. <role>webappsec</role>
>>>>>>>> 5. <name>WebAppSec</name>
>>>>>>>> 6. <enabled>true</enabled>
>>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>>> 8. </provider>
>>>>>>>> 9. <provider>
>>>>>>>> 10. <role>federation</role>
>>>>>>>> 11. <name>pac4j</name>
>>>>>>>> 12. <enabled>true</enabled>
>>>>>>>> 13. <param>
>>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>>> 16. </param>
>>>>>>>> 17. <param>
>>>>>>>> 18. <name>clientName</name>
>>>>>>>> 19. <value>OidcClient</value>
>>>>>>>> 20. </param>
>>>>>>>> 21. <param>
>>>>>>>> 22. <name>oidc.id</name>
>>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>>> 24. </param>
>>>>>>>> 25. <param>
>>>>>>>> 26. <name>oidc.secret</name>
>>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>>> 28. </param>
>>>>>>>> 29. <param>
>>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>>> 32. </param>
>>>>>>>> 33. </provider>
>>>>>>>> 34. <provider>
>>>>>>>> 35. <role>identity-assertion</role>
>>>>>>>> 36. <name>Default</name>
>>>>>>>> 37. <enabled>true</enabled>
>>>>>>>> 38. </provider>
>>>>>>>> 39. </gateway>
>>>>>>>> 40. <application>
>>>>>>>> 41. <name>knoxauth</name>
>>>>>>>> 42. </application>
>>>>>>>> 43. <service>
>>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>>> 45. <param>
>>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>>> 47. <value>false</value>
>>>>>>>> 48. </param>
>>>>>>>> 49. <param>
>>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>>> 51. <value>30000</value>
>>>>>>>> 52. </param>
>>>>>>>> 53. <param>
>>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>>> 56. </param>
>>>>>>>> 57. </service>
>>>>>>>> 58. </topology>
>>>>>>>>
>>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>>
>>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>>
>>>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>>>> to redirection!
>>>>>>>>
>>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>>> appreciated.
>>>>>>>> Regards
>>>>>>>> Nisha
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> Regards
>>>>> Nisha
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Colm O hEigeartaigh
>>>
>>> Talend Community Coder
>>> http://coders.talend.com
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Colm O hEigeartaigh <co...@apache.org>.
OK I'll check it. A question though - where is the redirection via
originalUrl supposed to take place? From the source I can see "originalUrl"
is referenced in:
gateway-applications/src/main/resources/applications/knoxauth/app/js/knoxauth.js
gateway-applications/src/main/resources/applications/knoxauth/app/redirecting.jsp
However, I'm not using the knoxauth application (with pac4j)?
Colm.
On Mon, Feb 19, 2018 at 6:04 PM, larry mccay <lm...@apache.org> wrote:
> Hi Colm -
>
> Thanks for trying to reproduce.
> Starting to get concerned about a regression....
>
> I see that you are using localhost - that will definitely present problems
> for Set-Cookie with some browsers.
> I know that Chrome will not accept it for sure.
>
> Can you switch to a full domain name?
> I usually just map localhost to www.local.com in my /etc/hosts file.
>
> You will also need to make sure to set the whitelist for the domain name.
>
> thanks,
>
> --larry
>
>
> On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <coheigea@apache.org
> > wrote:
>
>> I can reproduce the issue with Google. From the logs I see the following:
>>
>> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
>> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
>> https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?
>> originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/
>> webhdfs/v1/data/LICENSE.txt?op=OPEN
>> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
>> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
>> null
>> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
>> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
>> redirectUrl: https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?
>> originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/
>> webhdfs/v1/data/LICENSE.txt?op=OPEN
>> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
>> - Received request: GET /api/v1/websso
>>
>> It is getting an access token correctly and trying to redirect back to
>> the original URL. However from the logs above it seems to never hit the
>> original URL but instead hits the "redirectUrl". Is some parsing supposed
>> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
>> parameter and redirect to this instead?
>>
>> Colm.
>>
>> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>>> If it isn't being set, it could be that it isn't getting past the pac4j
>>> federation provider to the KnoxSSO service itself because there is a
>>> redirect from the pac4j provider or the Set-Cookie just isn't being
>>> accepted by the browser.
>>>
>>> The audit log does seem to be getting a redirect from pac4j.
>>>
>>> I haven't seen any examples of it working AAD - so we are in unchartered
>>> waters here.
>>>
>>> @Jerome - any insights?
>>>
>>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hello Larry,
>>>>
>>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>>> In SSO provider with pac4j, I can see cookies like:
>>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>>> pac4j.session.oidcNonceAttribute etc.
>>>>
>>>> IP address and hostnames are mapped, else the *basic auth* also would
>>>> have failed.
>>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>>
>>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org>
>>>> wrote:
>>>>
>>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>>> webhdfs.
>>>>> In order to rule out cookie related issue can you do a couple things:
>>>>>
>>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>>> browser
>>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>>> something that you can reference - I use www.local.com to map to
>>>>> localhost
>>>>>
>>>>> I think that ip address should work for this case actually but there
>>>>> are differences in browsers that might not let it.
>>>>> Also, if you had another service on another ip address, the browser
>>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>>
>>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Nisha,
>>>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>>>> the logs to debug and share them along with the audit log that would help
>>>>>> us to understand the problem better.
>>>>>>
>>>>>> Best,
>>>>>> Sandeep
>>>>>>
>>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <nisha.menon16@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>>
>>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>>
>>>>>>> *Details:*
>>>>>>>
>>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>>
>>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>>
>>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>>> session and state variables passed as below:
>>>>>>>
>>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>>
>>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>>
>>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>>
>>>>>>> After this, step 1 and 2 alternate several times and finally lands
>>>>>>> up in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>>
>>>>>>> This is my knoxsso.xml:
>>>>>>>
>>>>>>>
>>>>>>> 1. <topology>
>>>>>>> 2. <gateway>
>>>>>>> 3. <provider>
>>>>>>> 4. <role>webappsec</role>
>>>>>>> 5. <name>WebAppSec</name>
>>>>>>> 6. <enabled>true</enabled>
>>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>>> 8. </provider>
>>>>>>> 9. <provider>
>>>>>>> 10. <role>federation</role>
>>>>>>> 11. <name>pac4j</name>
>>>>>>> 12. <enabled>true</enabled>
>>>>>>> 13. <param>
>>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>>> 16. </param>
>>>>>>> 17. <param>
>>>>>>> 18. <name>clientName</name>
>>>>>>> 19. <value>OidcClient</value>
>>>>>>> 20. </param>
>>>>>>> 21. <param>
>>>>>>> 22. <name>oidc.id</name>
>>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>>> 24. </param>
>>>>>>> 25. <param>
>>>>>>> 26. <name>oidc.secret</name>
>>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>>> 28. </param>
>>>>>>> 29. <param>
>>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>>> 32. </param>
>>>>>>> 33. </provider>
>>>>>>> 34. <provider>
>>>>>>> 35. <role>identity-assertion</role>
>>>>>>> 36. <name>Default</name>
>>>>>>> 37. <enabled>true</enabled>
>>>>>>> 38. </provider>
>>>>>>> 39. </gateway>
>>>>>>> 40. <application>
>>>>>>> 41. <name>knoxauth</name>
>>>>>>> 42. </application>
>>>>>>> 43. <service>
>>>>>>> 44. <role>KNOXSSO</role>
>>>>>>> 45. <param>
>>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>>> 47. <value>false</value>
>>>>>>> 48. </param>
>>>>>>> 49. <param>
>>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>>> 51. <value>30000</value>
>>>>>>> 52. </param>
>>>>>>> 53. <param>
>>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>>> 56. </param>
>>>>>>> 57. </service>
>>>>>>> 58. </topology>
>>>>>>>
>>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>>
>>>>>>> gateway-audit.log when redirection error starts:
>>>>>>>
>>>>>>>
>>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>>
>>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>>> to redirection!
>>>>>>>
>>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>>> appreciated.
>>>>>>> Regards
>>>>>>> Nisha
>>>>>>>
>>>>>>>
>>>>>>
>>>>> Regards
>>>> Nisha
>>>>
>>>
>>>
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
Hi Colm -
Thanks for trying to reproduce.
Starting to get concerned about a regression....
I see that you are using localhost - that will definitely present problems
for Set-Cookie with some browsers.
I know that Chrome will not accept it for sure.
Can you switch to a full domain name?
I usually just map localhost to www.local.com in my /etc/hosts file.
You will also need to make sure to set the whitelist for the domain name.
thanks,
--larry
On Mon, Feb 19, 2018 at 12:49 PM, Colm O hEigeartaigh <co...@apache.org>
wrote:
> I can reproduce the issue with Google. From the logs I see the following:
>
> 2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
> https://localhost:8443/gateway/knoxssopac4j/api/v1/
> websso?originalUrl=https://localhost:8443/gateway/
> sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
> 2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
> (KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl =
> null
> 2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
> (DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
> redirectUrl: https://localhost:8443/gateway/knoxssopac4j/api/v1/
> websso?originalUrl=https://localhost:8443/gateway/
> sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
> 2018-02-19 17:21:58,488 DEBUG knox.gateway (GatewayFilter.java:doFilter(119))
> - Received request: GET /api/v1/websso
>
> It is getting an access token correctly and trying to redirect back to the
> original URL. However from the logs above it seems to never hit the
> original URL but instead hits the "redirectUrl". Is some parsing supposed
> to take place to extract "originalUrl" from the "pac4jRequestedUrl"
> parameter and redirect to this instead?
>
> Colm.
>
> On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
>
>> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
>> If it isn't being set, it could be that it isn't getting past the pac4j
>> federation provider to the KnoxSSO service itself because there is a
>> redirect from the pac4j provider or the Set-Cookie just isn't being
>> accepted by the browser.
>>
>> The audit log does seem to be getting a redirect from pac4j.
>>
>> I haven't seen any examples of it working AAD - so we are in unchartered
>> waters here.
>>
>> @Jerome - any insights?
>>
>> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello Larry,
>>>
>>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>>> In SSO provider with pac4j, I can see cookies like:
>>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>>> pac4j.session.oidcNonceAttribute etc.
>>>
>>> IP address and hostnames are mapped, else the *basic auth* also would
>>> have failed.
>>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>>
>>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> It looks like you may be using ip addresses for your Knox URLs - to
>>>> webhdfs.
>>>> In order to rule out cookie related issue can you do a couple things:
>>>>
>>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>>> browser
>>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>>> something that you can reference - I use www.local.com to map to
>>>> localhost
>>>>
>>>> I think that ip address should work for this case actually but there
>>>> are differences in browsers that might not let it.
>>>> Also, if you had another service on another ip address, the browser
>>>> would not present the cookie - so this is good to be aware of anyway.
>>>>
>>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello Nisha,
>>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>>> the logs to debug and share them along with the audit log that would help
>>>>> us to understand the problem better.
>>>>>
>>>>> Best,
>>>>> Sandeep
>>>>>
>>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>>
>>>>>> However, after the authentication at Azure login page, it gets into
>>>>>> an infinite loop and does not give back the original REST call response.
>>>>>>
>>>>>> *Details:*
>>>>>>
>>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>>
>>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>>
>>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>>> session and state variables passed as below:
>>>>>>
>>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>>
>>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>>
>>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>>
>>>>>> After this, step 1 and 2 alternate several times and finally lands up
>>>>>> in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>>
>>>>>> This is my knoxsso.xml:
>>>>>>
>>>>>>
>>>>>> 1. <topology>
>>>>>> 2. <gateway>
>>>>>> 3. <provider>
>>>>>> 4. <role>webappsec</role>
>>>>>> 5. <name>WebAppSec</name>
>>>>>> 6. <enabled>true</enabled>
>>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>>> 8. </provider>
>>>>>> 9. <provider>
>>>>>> 10. <role>federation</role>
>>>>>> 11. <name>pac4j</name>
>>>>>> 12. <enabled>true</enabled>
>>>>>> 13. <param>
>>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>>> 16. </param>
>>>>>> 17. <param>
>>>>>> 18. <name>clientName</name>
>>>>>> 19. <value>OidcClient</value>
>>>>>> 20. </param>
>>>>>> 21. <param>
>>>>>> 22. <name>oidc.id</name>
>>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>>> 24. </param>
>>>>>> 25. <param>
>>>>>> 26. <name>oidc.secret</name>
>>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>>> 28. </param>
>>>>>> 29. <param>
>>>>>> 30. <name>oidc.discoveryUri</name>
>>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>>> 32. </param>
>>>>>> 33. </provider>
>>>>>> 34. <provider>
>>>>>> 35. <role>identity-assertion</role>
>>>>>> 36. <name>Default</name>
>>>>>> 37. <enabled>true</enabled>
>>>>>> 38. </provider>
>>>>>> 39. </gateway>
>>>>>> 40. <application>
>>>>>> 41. <name>knoxauth</name>
>>>>>> 42. </application>
>>>>>> 43. <service>
>>>>>> 44. <role>KNOXSSO</role>
>>>>>> 45. <param>
>>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>>> 47. <value>false</value>
>>>>>> 48. </param>
>>>>>> 49. <param>
>>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>>> 51. <value>30000</value>
>>>>>> 52. </param>
>>>>>> 53. <param>
>>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>>> 56. </param>
>>>>>> 57. </service>
>>>>>> 58. </topology>
>>>>>>
>>>>>> I tried using response_type "id_token", enabling nonces,
>>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>>
>>>>>> gateway-audit.log when redirection error starts:
>>>>>>
>>>>>>
>>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>>
>>>>>> It clearly shows Response status as "302" and not "200". This leads
>>>>>> to redirection!
>>>>>>
>>>>>> What could I be missing here? Any pointers will be greatly
>>>>>> appreciated.
>>>>>> Regards
>>>>>> Nisha
>>>>>>
>>>>>>
>>>>>
>>>> Regards
>>> Nisha
>>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Colm O hEigeartaigh <co...@apache.org>.
I can reproduce the issue with Google. From the logs I see the following:
2018-02-19 17:21:58,484 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(91)) - Get from session: pac4jRequestedUrl =
https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
2018-02-19 17:21:58,485 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(107)) - Save in session: pac4jRequestedUrl = null
2018-02-19 17:21:58,485 DEBUG engine.DefaultCallbackLogic
(DefaultCallbackLogic.java:redirectToOriginallyRequestedUrl(137)) -
redirectUrl:
https://localhost:8443/gateway/knoxssopac4j/api/v1/websso?originalUrl=https://localhost:8443/gateway/sandbox-ssopac4j/webhdfs/v1/data/LICENSE.txt?op=OPEN
2018-02-19 17:21:58,488 DEBUG knox.gateway
(GatewayFilter.java:doFilter(119)) - Received request: GET /api/v1/websso
It is getting an access token correctly and trying to redirect back to the
original URL. However from the logs above it seems to never hit the
original URL but instead hits the "redirectUrl". Is some parsing supposed
to take place to extract "originalUrl" from the "pac4jRequestedUrl"
parameter and redirect to this instead?
Colm.
On Mon, Feb 19, 2018 at 4:16 PM, larry mccay <lm...@apache.org> wrote:
> No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
> If it isn't being set, it could be that it isn't getting past the pac4j
> federation provider to the KnoxSSO service itself because there is a
> redirect from the pac4j provider or the Set-Cookie just isn't being
> accepted by the browser.
>
> The audit log does seem to be getting a redirect from pac4j.
>
> I haven't seen any examples of it working AAD - so we are in unchartered
> waters here.
>
> @Jerome - any insights?
>
> On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello Larry,
>>
>> hadoop-jwt cookie is not set. Isnt this for JWT provider?
>> In SSO provider with pac4j, I can see cookies like:
>> pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
>> pac4j.session.oidcNonceAttribute etc.
>>
>> IP address and hostnames are mapped, else the *basic auth* also would
>> have failed.
>> My issue is only when I use pac4j with Oidc client and Azure AD.
>>
>> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
>>
>>> It looks like you may be using ip addresses for your Knox URLs - to
>>> webhdfs.
>>> In order to rule out cookie related issue can you do a couple things:
>>>
>>> 1. check whether a cookie called hadoop-jwt is actually set in your
>>> browser
>>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>>> something that you can reference - I use www.local.com to map to
>>> localhost
>>>
>>> I think that ip address should work for this case actually but there are
>>> differences in browsers that might not let it.
>>> Also, if you had another service on another ip address, the browser
>>> would not present the cookie - so this is good to be aware of anyway.
>>>
>>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>>> wrote:
>>>
>>>> Hello Nisha,
>>>> Can you share details of "mycluster" topology ? also, can you turn up
>>>> the logs to debug and share them along with the audit log that would help
>>>> us to understand the problem better.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>>
>>>>> However, after the authentication at Azure login page, it gets into an
>>>>> infinite loop and does not give back the original REST call response.
>>>>>
>>>>> *Details:*
>>>>>
>>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>>
>>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>>
>>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>>> session and state variables passed as below:
>>>>>
>>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>>
>>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>>> <http://login.microsoftonline.com/>* like below:
>>>>>
>>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>>
>>>>> After this, step 1 and 2 alternate several times and finally lands up
>>>>> in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>>
>>>>> This is my knoxsso.xml:
>>>>>
>>>>>
>>>>> 1. <topology>
>>>>> 2. <gateway>
>>>>> 3. <provider>
>>>>> 4. <role>webappsec</role>
>>>>> 5. <name>WebAppSec</name>
>>>>> 6. <enabled>true</enabled>
>>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>>> 8. </provider>
>>>>> 9. <provider>
>>>>> 10. <role>federation</role>
>>>>> 11. <name>pac4j</name>
>>>>> 12. <enabled>true</enabled>
>>>>> 13. <param>
>>>>> 14. <name>pac4j.callbackUrl</name>
>>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>>> 16. </param>
>>>>> 17. <param>
>>>>> 18. <name>clientName</name>
>>>>> 19. <value>OidcClient</value>
>>>>> 20. </param>
>>>>> 21. <param>
>>>>> 22. <name>oidc.id</name>
>>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>>> 24. </param>
>>>>> 25. <param>
>>>>> 26. <name>oidc.secret</name>
>>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>>> 28. </param>
>>>>> 29. <param>
>>>>> 30. <name>oidc.discoveryUri</name>
>>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>>> 32. </param>
>>>>> 33. </provider>
>>>>> 34. <provider>
>>>>> 35. <role>identity-assertion</role>
>>>>> 36. <name>Default</name>
>>>>> 37. <enabled>true</enabled>
>>>>> 38. </provider>
>>>>> 39. </gateway>
>>>>> 40. <application>
>>>>> 41. <name>knoxauth</name>
>>>>> 42. </application>
>>>>> 43. <service>
>>>>> 44. <role>KNOXSSO</role>
>>>>> 45. <param>
>>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>>> 47. <value>false</value>
>>>>> 48. </param>
>>>>> 49. <param>
>>>>> 50. <name>knoxsso.token.ttl</name>
>>>>> 51. <value>30000</value>
>>>>> 52. </param>
>>>>> 53. <param>
>>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>>> 56. </param>
>>>>> 57. </service>
>>>>> 58. </topology>
>>>>>
>>>>> I tried using response_type "id_token", enabling nonces,
>>>>> knoxsso.secure to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>>
>>>>> gateway-audit.log when redirection error starts:
>>>>>
>>>>>
>>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>>
>>>>> It clearly shows Response status as "302" and not "200". This leads to
>>>>> redirection!
>>>>>
>>>>> What could I be missing here? Any pointers will be greatly appreciated.
>>>>> Regards
>>>>> Nisha
>>>>>
>>>>>
>>>>
>>> Regards
>> Nisha
>>
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
No, the hadoop-jwt cookie is for KnoxSSO and the SSOCookieProvider.
If it isn't being set, it could be that it isn't getting past the pac4j
federation provider to the KnoxSSO service itself because there is a
redirect from the pac4j provider or the Set-Cookie just isn't being
accepted by the browser.
The audit log does seem to be getting a redirect from pac4j.
I haven't seen any examples of it working AAD - so we are in unchartered
waters here.
@Jerome - any insights?
On Mon, Feb 19, 2018 at 2:04 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Hello Larry,
>
> hadoop-jwt cookie is not set. Isnt this for JWT provider?
> In SSO provider with pac4j, I can see cookies like: pac4j.session.pac4jRequestedUrl,
> pac4j.session.oidcStateAttribute, pac4j.session.oidcNonceAttribute etc.
>
> IP address and hostnames are mapped, else the *basic auth* also would
> have failed.
> My issue is only when I use pac4j with Oidc client and Azure AD.
>
> On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
>
>> It looks like you may be using ip addresses for your Knox URLs - to
>> webhdfs.
>> In order to rule out cookie related issue can you do a couple things:
>>
>> 1. check whether a cookie called hadoop-jwt is actually set in your
>> browser
>> 2. if not, you may want to set an actual domain in your /etc/hosts or
>> something that you can reference - I use www.local.com to map to
>> localhost
>>
>> I think that ip address should work for this case actually but there are
>> differences in browsers that might not let it.
>> Also, if you had another service on another ip address, the browser would
>> not present the cookie - so this is good to be aware of anyway.
>>
>> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
>> wrote:
>>
>>> Hello Nisha,
>>> Can you share details of "mycluster" topology ? also, can you turn up
>>> the logs to debug and share them along with the audit log that would help
>>> us to understand the problem better.
>>>
>>> Best,
>>> Sandeep
>>>
>>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>>
>>>> However, after the authentication at Azure login page, it gets into an
>>>> infinite loop and does not give back the original REST call response.
>>>>
>>>> *Details:*
>>>>
>>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>>
>>>> 2. It redirects to *https://**login.microsoftonline.com
>>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>>
>>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code,
>>>> session and state variables passed as below:
>>>>
>>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>>
>>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>>> <http://login.microsoftonline.com/>* like below:
>>>>
>>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>>
>>>> After this, step 1 and 2 alternate several times and finally lands up
>>>> in "ERR_TOO_MANY_REDIRECTS"!!!
>>>>
>>>> This is my knoxsso.xml:
>>>>
>>>>
>>>> 1. <topology>
>>>> 2. <gateway>
>>>> 3. <provider>
>>>> 4. <role>webappsec</role>
>>>> 5. <name>WebAppSec</name>
>>>> 6. <enabled>true</enabled>
>>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>>> 8. </provider>
>>>> 9. <provider>
>>>> 10. <role>federation</role>
>>>> 11. <name>pac4j</name>
>>>> 12. <enabled>true</enabled>
>>>> 13. <param>
>>>> 14. <name>pac4j.callbackUrl</name>
>>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>>> 16. </param>
>>>> 17. <param>
>>>> 18. <name>clientName</name>
>>>> 19. <value>OidcClient</value>
>>>> 20. </param>
>>>> 21. <param>
>>>> 22. <name>oidc.id</name>
>>>> 23. <value>385c2bc*****************2695eaa34</value>
>>>> 24. </param>
>>>> 25. <param>
>>>> 26. <name>oidc.secret</name>
>>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>>> 28. </param>
>>>> 29. <param>
>>>> 30. <name>oidc.discoveryUri</name>
>>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>>> 32. </param>
>>>> 33. </provider>
>>>> 34. <provider>
>>>> 35. <role>identity-assertion</role>
>>>> 36. <name>Default</name>
>>>> 37. <enabled>true</enabled>
>>>> 38. </provider>
>>>> 39. </gateway>
>>>> 40. <application>
>>>> 41. <name>knoxauth</name>
>>>> 42. </application>
>>>> 43. <service>
>>>> 44. <role>KNOXSSO</role>
>>>> 45. <param>
>>>> 46. <name>knoxsso.cookie.secure.only</name>
>>>> 47. <value>false</value>
>>>> 48. </param>
>>>> 49. <param>
>>>> 50. <name>knoxsso.token.ttl</name>
>>>> 51. <value>30000</value>
>>>> 52. </param>
>>>> 53. <param>
>>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>>> 56. </param>
>>>> 57. </service>
>>>> 58. </topology>
>>>>
>>>> I tried using response_type "id_token", enabling nonces, knoxsso.secure
>>>> to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>>
>>>> gateway-audit.log when redirection error starts:
>>>>
>>>>
>>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>>
>>>> It clearly shows Response status as "302" and not "200". This leads to
>>>> redirection!
>>>>
>>>> What could I be missing here? Any pointers will be greatly appreciated.
>>>> Regards
>>>> Nisha
>>>>
>>>>
>>>
>> Regards
> Nisha
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hello Larry,
hadoop-jwt cookie is not set. Isnt this for JWT provider?
In SSO provider with pac4j, I can see cookies like:
pac4j.session.pac4jRequestedUrl, pac4j.session.oidcStateAttribute,
pac4j.session.oidcNonceAttribute etc.
IP address and hostnames are mapped, else the *basic auth* also would have
failed.
My issue is only when I use pac4j with Oidc client and Azure AD.
On Fri, Feb 16, 2018 at 10:11 PM, larry mccay <lm...@apache.org> wrote:
> It looks like you may be using ip addresses for your Knox URLs - to
> webhdfs.
> In order to rule out cookie related issue can you do a couple things:
>
> 1. check whether a cookie called hadoop-jwt is actually set in your browser
> 2. if not, you may want to set an actual domain in your /etc/hosts or
> something that you can reference - I use www.local.com to map to localhost
>
> I think that ip address should work for this case actually but there are
> differences in browsers that might not let it.
> Also, if you had another service on another ip address, the browser would
> not present the cookie - so this is good to be aware of anyway.
>
> On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Nisha,
>> Can you share details of "mycluster" topology ? also, can you turn up the
>> logs to debug and share them along with the audit log that would help us to
>> understand the problem better.
>>
>> Best,
>> Sandeep
>>
>> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> I have setup KNOX to connect with Azure AD using pac4j.
>>>
>>> However, after the authentication at Azure login page, it gets into an
>>> infinite loop and does not give back the original REST call response.
>>>
>>> *Details:*
>>>
>>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>>
>>> 2. It redirects to *https://**login.microsoftonline.com
>>> <http://login.microsoftonline.com/>* and asks for credentials.
>>>
>>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session
>>> and state variables passed as below:
>>>
>>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>>
>>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>>> <http://login.microsoftonline.com/>* like below:
>>>
>>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>>
>>> After this, step 1 and 2 alternate several times and finally lands up in
>>> "ERR_TOO_MANY_REDIRECTS"!!!
>>>
>>> This is my knoxsso.xml:
>>>
>>>
>>> 1. <topology>
>>> 2. <gateway>
>>> 3. <provider>
>>> 4. <role>webappsec</role>
>>> 5. <name>WebAppSec</name>
>>> 6. <enabled>true</enabled>
>>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>>> 8. </provider>
>>> 9. <provider>
>>> 10. <role>federation</role>
>>> 11. <name>pac4j</name>
>>> 12. <enabled>true</enabled>
>>> 13. <param>
>>> 14. <name>pac4j.callbackUrl</name>
>>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>>> 16. </param>
>>> 17. <param>
>>> 18. <name>clientName</name>
>>> 19. <value>OidcClient</value>
>>> 20. </param>
>>> 21. <param>
>>> 22. <name>oidc.id</name>
>>> 23. <value>385c2bc*****************2695eaa34</value>
>>> 24. </param>
>>> 25. <param>
>>> 26. <name>oidc.secret</name>
>>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>>> 28. </param>
>>> 29. <param>
>>> 30. <name>oidc.discoveryUri</name>
>>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>>> 32. </param>
>>> 33. </provider>
>>> 34. <provider>
>>> 35. <role>identity-assertion</role>
>>> 36. <name>Default</name>
>>> 37. <enabled>true</enabled>
>>> 38. </provider>
>>> 39. </gateway>
>>> 40. <application>
>>> 41. <name>knoxauth</name>
>>> 42. </application>
>>> 43. <service>
>>> 44. <role>KNOXSSO</role>
>>> 45. <param>
>>> 46. <name>knoxsso.cookie.secure.only</name>
>>> 47. <value>false</value>
>>> 48. </param>
>>> 49. <param>
>>> 50. <name>knoxsso.token.ttl</name>
>>> 51. <value>30000</value>
>>> 52. </param>
>>> 53. <param>
>>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>>> 56. </param>
>>> 57. </service>
>>> 58. </topology>
>>>
>>> I tried using response_type "id_token", enabling nonces, knoxsso.secure
>>> to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>>
>>> gateway-audit.log when redirection error starts:
>>>
>>>
>>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>>
>>> It clearly shows Response status as "302" and not "200". This leads to
>>> redirection!
>>>
>>> What could I be missing here? Any pointers will be greatly appreciated.
>>> Regards
>>> Nisha
>>>
>>>
>>
> Regards
Nisha
Re: KNOX Pac4j Azure AD Open ID
Posted by larry mccay <lm...@apache.org>.
It looks like you may be using ip addresses for your Knox URLs - to webhdfs.
In order to rule out cookie related issue can you do a couple things:
1. check whether a cookie called hadoop-jwt is actually set in your browser
2. if not, you may want to set an actual domain in your /etc/hosts or
something that you can reference - I use www.local.com to map to localhost
I think that ip address should work for this case actually but there are
differences in browsers that might not let it.
Also, if you had another service on another ip address, the browser would
not present the cookie - so this is good to be aware of anyway.
On Fri, Feb 16, 2018 at 8:55 AM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Nisha,
> Can you share details of "mycluster" topology ? also, can you turn up the
> logs to debug and share them along with the audit log that would help us to
> understand the problem better.
>
> Best,
> Sandeep
>
> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I have setup KNOX to connect with Azure AD using pac4j.
>>
>> However, after the authentication at Azure login page, it gets into an
>> infinite loop and does not give back the original REST call response.
>>
>> *Details:*
>>
>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>
>> 2. It redirects to *https://**login.microsoftonline.com
>> <http://login.microsoftonline.com/>* and asks for credentials.
>>
>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session
>> and state variables passed as below:
>>
>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>
>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>> <http://login.microsoftonline.com/>* like below:
>>
>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>
>> After this, step 1 and 2 alternate several times and finally lands up in
>> "ERR_TOO_MANY_REDIRECTS"!!!
>>
>> This is my knoxsso.xml:
>>
>>
>> 1. <topology>
>> 2. <gateway>
>> 3. <provider>
>> 4. <role>webappsec</role>
>> 5. <name>WebAppSec</name>
>> 6. <enabled>true</enabled>
>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>> 8. </provider>
>> 9. <provider>
>> 10. <role>federation</role>
>> 11. <name>pac4j</name>
>> 12. <enabled>true</enabled>
>> 13. <param>
>> 14. <name>pac4j.callbackUrl</name>
>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>> 16. </param>
>> 17. <param>
>> 18. <name>clientName</name>
>> 19. <value>OidcClient</value>
>> 20. </param>
>> 21. <param>
>> 22. <name>oidc.id</name>
>> 23. <value>385c2bc*****************2695eaa34</value>
>> 24. </param>
>> 25. <param>
>> 26. <name>oidc.secret</name>
>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>> 28. </param>
>> 29. <param>
>> 30. <name>oidc.discoveryUri</name>
>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>> 32. </param>
>> 33. </provider>
>> 34. <provider>
>> 35. <role>identity-assertion</role>
>> 36. <name>Default</name>
>> 37. <enabled>true</enabled>
>> 38. </provider>
>> 39. </gateway>
>> 40. <application>
>> 41. <name>knoxauth</name>
>> 42. </application>
>> 43. <service>
>> 44. <role>KNOXSSO</role>
>> 45. <param>
>> 46. <name>knoxsso.cookie.secure.only</name>
>> 47. <value>false</value>
>> 48. </param>
>> 49. <param>
>> 50. <name>knoxsso.token.ttl</name>
>> 51. <value>30000</value>
>> 52. </param>
>> 53. <param>
>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>> 56. </param>
>> 57. </service>
>> 58. </topology>
>>
>> I tried using response_type "id_token", enabling nonces, knoxsso.secure
>> to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>
>> gateway-audit.log when redirection error starts:
>>
>>
>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>
>> It clearly shows Response status as "302" and not "200". This leads to
>> redirection!
>>
>> What could I be missing here? Any pointers will be greatly appreciated.
>> Regards
>> Nisha
>>
>>
>
Re: KNOX Pac4j Azure AD Open ID
Posted by Nisha Menon <ni...@gmail.com>.
Hello Sandeep,
Sorry, that was an incomplete post earlier.
*myCluster.topology:*
1. <topology>
2. <gateway>
3. <provider>
4. <role>webappsec</role>
5. <name>WebAppSec</name>
6. <enabled>true</enabled>
7. <param>
<name>cors.enabled</name>
<value>true</value>
</param>
8. </provider>
9. <provider>
10. <role>federation</role>
11. <name>SSOCookieProvider</name>
<enabled>true</enabled>
<param>
<name>sso.authentication.provider.url</name>
<value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
<https://52.169.143.168:8442/gateway/knoxsso/api/v1/websso></value>
</param>
12. </provider>
13. <provider>
14. <role>identity-assertion</role>
15. <name>Default</name>
16. <enabled>true</enabled>
17. </provider>
18. </gateway>
19. <service>
<role>NAMENODE</role>
<url>hdfs://myHostname:8020
<http://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:8020/></url>
</service>
<service>
<role>JOBTRACKER</role>
<url>rpc://myHostname:8050
<http://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:8050/></url>
</service>
<service>
<role>WEBHDFS</role>
<url>http://myHostname:50070/webhdfs
<http://dap-m1.2mwungiz1ezu3cgpxipb55z2fh.fx.internal.cloudapp.net:50070/webhdfs></url>
</service>
20. </topology>
*Logs: *
gateway.log: Nothing appears when the call starts. The last message is:
2018-02-19 06:28:11,040 INFO hadoop.gateway
(GatewayServer.java:internalActivateTopology(566)) - Activating topology
knoxsso
2018-02-19 06:28:11,040 INFO hadoop.gateway
(GatewayServer.java:internalActivateArchive(576)) - Activating topology
knoxsso archive %2F
gateway-audit.log: Same as shared earlier:
18/02/19 06:30:01
||477ab97f-884b-4421-90f9-70b7925c01b9|audit|WEBHDFS||||access|uri|/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS|unavailable|Request
method: GET
18/02/19 06:30:01
||477ab97f-884b-4421-90f9-70b7925c01b9|audit|WEBHDFS||||access|uri|/gateway/
myCluster/webhdfs/v1/user?op=LISTSTATUS|success|Response status: 302
18/02/19 06:30:02
||61b0dff1-ca1a-4fc2-868b-2e7b5d771ed0|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=
https://52.169.143.168:8442/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS|unavailable|Request
method: GET
18/02/19 06:30:02
||61b0dff1-ca1a-4fc2-868b-2e7b5d771ed0|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=
https://52.169.143.168:8442/gateway/myCluster/webhdfs/v1/user?op=LISTSTATUS|success|Response
status: 302
18/02/19 06:31:03
||ea79e31f-00a6-4404-94d6-ddc22d7ab5dd|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzViBkITGagTwTx0uHenFkQaq-6glGCQqPd7qX0buT_7n0nNCrT1Sv3f8dwcv6OzoMFlU-LBynYYhbkxGUX82WndCl3nalcZMsL-PB8TNGjqYx6aJ0k634C6_-1LGdIGsVkTK3urUSozifWRQ8H5o_vfTp-94TMQEVhjIi8VznQaM5jyCXPckolSV8_-ux_tRJG4KDLUy7tSbPTDKSnZUt7uyr_xrwzohscct-_HLACuAUpdz6pl1x-ap6b_dbNTHrrurcDHrFCM5YtJzfxnUTtVzqq-EH1irGrAG7G0DI-cbUETF2RG9B_QaE7nZDpS02LFmKB7Smkyr3KzYaRbCOzYn8e3qxZ2AuXbCNt1s6VSL1lEs2CJgzPB6U39UC93XMxX-9Bm1JfUNs4G8JxUnMblu0yhLfEHNeDeDkg5GMKHA_1CnvudKYyWqR6Jcj4UaJb7bSosM6CK0NO7jGoy1ZGqi_CKc84gKOsGYOTnzC5uLH5vMPWVTNiChCeBrURq339ZwcA01ArcQZmBztEksibU-A_RzZ1ZqqnLxcnByAWhPMjXyF8GwdLkfcGg_qkMEicNIkE4gEdW06TAInh0pZr71V5BvzW2f3qDDN6L_0IAk-aItcsCFs1q8EADk1rdtMbcEyl-ZgLJQuVk7izwn7T2p1p1thL2wpbHQ3ahh_Re5wAnn_0HOCKFXoTZeWSJ3SMq-_Wq9ill8g01ySI2bzbZAuFQhGrodKPCxkYfKO2TcgAA&state=IagOJAVbIKeAQyXMIDjBWJGduIGzCUUPM4RxPVjiHoA&session_state=5bb1f616-d21c-4e70-bbfe-b4458c97f5f8|unavailable|Request
method: GET
18/02/19 06:31:03
||ea79e31f-00a6-4404-94d6-ddc22d7ab5dd|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzViBkITGagTwTx0uHenFkQaq-6glGCQqPd7qX0buT_7n0nNCrT1Sv3f8dwcv6OzoMFlU-LBynYYhbkxGUX82WndCl3nalcZMsL-PB8TNGjqYx6aJ0k634C6_-1LGdIGsVkTK3urUSozifWRQ8H5o_vfTp-94TMQEVhjIi8VznQaM5jyCXPckolSV8_-ux_tRJG4KDLUy7tSbPTDKSnZUt7uyr_xrwzohscct-_HLACuAUpdz6pl1x-ap6b_dbNTHrrurcDHrFCM5YtJzfxnUTtVzqq-EH1irGrAG7G0DI-cbUETF2RG9B_QaE7nZDpS02LFmKB7Smkyr3KzYaRbCOzYn8e3qxZ2AuXbCNt1s6VSL1lEs2CJgzPB6U39UC93XMxX-9Bm1JfUNs4G8JxUnMblu0yhLfEHNeDeDkg5GMKHA_1CnvudKYyWqR6Jcj4UaJb7bSosM6CK0NO7jGoy1ZGqi_CKc84gKOsGYOTnzC5uLH5vMPWVTNiChCeBrURq339ZwcA01ArcQZmBztEksibU-A_RzZ1ZqqnLxcnByAWhPMjXyF8GwdLkfcGg_qkMEicNIkE4gEdW06TAInh0pZr71V5BvzW2f3qDDN6L_0IAk-aItcsCFs1q8EADk1rdtMbcEyl-ZgLJQuVk7izwn7T2p1p1thL2wpbHQ3ahh_Re5wAnn_0HOCKFXoTZeWSJ3SMq-_Wq9ill8g01ySI2bzbZAuFQhGrodKPCxkYfKO2TcgAA&state=IagOJAVbIKeAQyXMIDjBWJGduIGzCUUPM4RxPVjiHoA&session_state=5bb1f616-d21c-4e70-bbfe-b4458c97f5f8|success|Response
status: 302
18/02/19 06:31:04
||a1359a64-b95e-4ba2-b7da-b034e7a5aaef|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzNeoHuSgOIUpFOWWu5xR4oKQv5EMmCyrpR7TqamIS_pvoLeIJyZ3TUZQ09MuUzsyfhu2edGE4kZGdeFApDzU9rkB9548qwrsC4uNeK33suYdcBxMCf8V_T5jTE3vwrfL6XLASVYqCFpvdh8E73F6OmIvPY1CZIuhpoys0hZ_ENcifFW_k1heBkJ7RQEsWPos6r0ySGDrYb6K8FU--MPGCCCGEXyUsqA8winTLoUOFKKSqU5KPpUTeBNiUM5NfaKQ4tfk3WAsVbMhB-ylwctwd36-MzvFVXSPS6oRyc4Qzif2EJucFk34v2Uk8lO1WCa3vCKMzHI_gwPFNB7WqIfguGxgVHx714bWIGh86LxWstVxyX9g5MI2t0kp1VmT06sxy_TRKNNuveUuaNRa65NVSuxNoSCfJR846Ot5mWZfOVGeWPtSZZTgYpSaIAXduoNJ2kdz8rfoGRvJJCLMEJzYto7LcNIAYdu44FNunSa85mPpJmMqGM82OTmHLCNiruuVMYb4i4mULmRbP_HfrCW6CEuQTSKgWl157ntdf_dTfH9BqdMMF3oDBDFJ9_0BlfLB7Ca8a9Nxuf42iW1IZGjYHRg38IshGKza6Bx-aIbCmQafYwvu2qJglA4zv8FavBPWBaQBOAbXUiG5YeKO0iZjVqfcjYs4DhjFC5NXrMVuAvgc5mYWe7tgVr9MuXHpBgLAKFTyJzxh3KHZauNBCTn_l5nnbNLLWZX6gCeQiXaNme8IgAA&state=37Fkr9XwLsNs2cIs3FrFnqt1tnX2FYAks2PWT9QsUiw&session_state=5bb1f616-d21c-4e70-bbfe-b4458c97f5f8|unavailable|Request
method: GET
18/02/19 06:31:04
||a1359a64-b95e-4ba2-b7da-b034e7a5aaef|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzNeoHuSgOIUpFOWWu5xR4oKQv5EMmCyrpR7TqamIS_pvoLeIJyZ3TUZQ09MuUzsyfhu2edGE4kZGdeFApDzU9rkB9548qwrsC4uNeK33suYdcBxMCf8V_T5jTE3vwrfL6XLASVYqCFpvdh8E73F6OmIvPY1CZIuhpoys0hZ_ENcifFW_k1heBkJ7RQEsWPos6r0ySGDrYb6K8FU--MPGCCCGEXyUsqA8winTLoUOFKKSqU5KPpUTeBNiUM5NfaKQ4tfk3WAsVbMhB-ylwctwd36-MzvFVXSPS6oRyc4Qzif2EJucFk34v2Uk8lO1WCa3vCKMzHI_gwPFNB7WqIfguGxgVHx714bWIGh86LxWstVxyX9g5MI2t0kp1VmT06sxy_TRKNNuveUuaNRa65NVSuxNoSCfJR846Ot5mWZfOVGeWPtSZZTgYpSaIAXduoNJ2kdz8rfoGRvJJCLMEJzYto7LcNIAYdu44FNunSa85mPpJmMqGM82OTmHLCNiruuVMYb4i4mULmRbP_HfrCW6CEuQTSKgWl157ntdf_dTfH9BqdMMF3oDBDFJ9_0BlfLB7Ca8a9Nxuf42iW1IZGjYHRg38IshGKza6Bx-aIbCmQafYwvu2qJglA4zv8FavBPWBaQBOAbXUiG5YeKO0iZjVqfcjYs4DhjFC5NXrMVuAvgc5mYWe7tgVr9MuXHpBgLAKFTyJzxh3KHZauNBCTn_l5nnbNLLWZX6gCeQiXaNme8IgAA&state=37Fkr9XwLsNs2cIs3FrFnqt1tnX2FYAks2PWT9QsUiw&session_state=5bb1f616-d21c-4e70-bbfe-b4458c97f5f8|success|Response
status: 302
This keeps continuing till the page loads error on browser.
Regards,
Nisha
On Fri, Feb 16, 2018 at 7:25 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Nisha,
> Can you share details of "mycluster" topology ? also, can you turn up the
> logs to debug and share them along with the audit log that would help us to
> understand the problem better.
>
> Best,
> Sandeep
>
> On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I have setup KNOX to connect with Azure AD using pac4j.
>>
>> However, after the authentication at Azure login page, it gets into an
>> infinite loop and does not give back the original REST call response.
>>
>> *Details:*
>>
>> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
>> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>>
>> 2. It redirects to *https://**login.microsoftonline.com
>> <http://login.microsoftonline.com/>* and asks for credentials.
>>
>> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
>> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session
>> and state variables passed as below:
>>
>> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
>> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>>
>> 2. Following this call, it *again *calls the *login.microsoftonline.com
>> <http://login.microsoftonline.com/>* like below:
>>
>> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
>> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>>
>> After this, step 1 and 2 alternate several times and finally lands up in
>> "ERR_TOO_MANY_REDIRECTS"!!!
>>
>> This is my knoxsso.xml:
>>
>>
>> 1. <topology>
>> 2. <gateway>
>> 3. <provider>
>> 4. <role>webappsec</role>
>> 5. <name>WebAppSec</name>
>> 6. <enabled>true</enabled>
>> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
>> 8. </provider>
>> 9. <provider>
>> 10. <role>federation</role>
>> 11. <name>pac4j</name>
>> 12. <enabled>true</enabled>
>> 13. <param>
>> 14. <name>pac4j.callbackUrl</name>
>> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
>> 16. </param>
>> 17. <param>
>> 18. <name>clientName</name>
>> 19. <value>OidcClient</value>
>> 20. </param>
>> 21. <param>
>> 22. <name>oidc.id</name>
>> 23. <value>385c2bc*****************2695eaa34</value>
>> 24. </param>
>> 25. <param>
>> 26. <name>oidc.secret</name>
>> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
>> 28. </param>
>> 29. <param>
>> 30. <name>oidc.discoveryUri</name>
>> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
>> 32. </param>
>> 33. </provider>
>> 34. <provider>
>> 35. <role>identity-assertion</role>
>> 36. <name>Default</name>
>> 37. <enabled>true</enabled>
>> 38. </provider>
>> 39. </gateway>
>> 40. <application>
>> 41. <name>knoxauth</name>
>> 42. </application>
>> 43. <service>
>> 44. <role>KNOXSSO</role>
>> 45. <param>
>> 46. <name>knoxsso.cookie.secure.only</name>
>> 47. <value>false</value>
>> 48. </param>
>> 49. <param>
>> 50. <name>knoxsso.token.ttl</name>
>> 51. <value>30000</value>
>> 52. </param>
>> 53. <param>
>> 54. <name>knoxsso.redirect.whitelist.regex</name>
>> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
>> 56. </param>
>> 57. </service>
>> 58. </topology>
>>
>> I tried using response_type "id_token", enabling nonces, knoxsso.secure
>> to true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>>
>> gateway-audit.log when redirection error starts:
>>
>>
>> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>>
>> It clearly shows Response status as "302" and not "200". This leads to
>> redirection!
>>
>> What could I be missing here? Any pointers will be greatly appreciated.
>> Regards
>> Nisha
>>
>>
Re: KNOX Pac4j Azure AD Open ID
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Nisha,
Can you share details of "mycluster" topology ? also, can you turn up the
logs to debug and share them along with the audit log that would help us to
understand the problem better.
Best,
Sandeep
On Fri, Feb 16, 2018 at 3:16 AM, Nisha Menon <ni...@gmail.com>
wrote:
> Hello,
>
> I have setup KNOX to connect with Azure AD using pac4j.
>
> However, after the authentication at Azure login page, it gets into an
> infinite loop and does not give back the original REST call response.
>
> *Details:*
>
> 1. I try to access the original URL eg: *https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS
> <https://x.x.2.3:8442/gateway/mycluster/webhdfs/v1/user?op=LISTSTATUS>*
>
> 2. It redirects to *https://**login.microsoftonline.com
> <http://login.microsoftonline.com/>* and asks for credentials.
>
> 3. After successful login at Azure login page, it redirects to *http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso
> <http://x.x.2.3:8442/gateway/knoxsso/api/v1/websso>* with code, session
> and state variables passed as below:
>
> *https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18
> <https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4k***********************LFFm7C9cIShE7nggAA&state=5dzTZBYhEVDBrA*****************GZRNfANGb5ls&session_state=42f2447b-621***********790eaa2d18>*
>
> 2. Following this call, it *again *calls the *login.microsoftonline.com
> <http://login.microsoftonline.com/>* like below:
>
> *https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient≻ope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw
> <https://login.microsoftonline.com/f82969ba-b***********c1d0557a/oauth2/authorize?response_type=code&client_id=385*******3-a4bdceaa34&redirect_uri=https%3A%2F%2Fx.x.2.3%3A8442%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso%3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient%E2%89%BBope=openid+profile+email&state=5dzTZBYhEVDBrAInao9VHDRd33uiRp-GZRNfANGb5ls&nonce=BvCUroM7_aKFjmbLYxaxbS0Mq9SJ8If0CUpITEGB-bw>*
>
> After this, step 1 and 2 alternate several times and finally lands up in
> "ERR_TOO_MANY_REDIRECTS"!!!
>
> This is my knoxsso.xml:
>
>
> 1. <topology>
> 2. <gateway>
> 3. <provider>
> 4. <role>webappsec</role>
> 5. <name>WebAppSec</name>
> 6. <enabled>true</enabled>
> 7. <param><name>xframe.options.enabled</name><value>true</value></param>
> 8. </provider>
> 9. <provider>
> 10. <role>federation</role>
> 11. <name>pac4j</name>
> 12. <enabled>true</enabled>
> 13. <param>
> 14. <name>pac4j.callbackUrl</name>
> 15. <value>https://x.x.2.3:8442/gateway/knoxsso/api/v1/websso</value>
> 16. </param>
> 17. <param>
> 18. <name>clientName</name>
> 19. <value>OidcClient</value>
> 20. </param>
> 21. <param>
> 22. <name>oidc.id</name>
> 23. <value>385c2bc*****************2695eaa34</value>
> 24. </param>
> 25. <param>
> 26. <name>oidc.secret</name>
> 27. <value>Y30wOwM88BY************vYmPp8KMyDY2W+o=</value>
> 28. </param>
> 29. <param>
> 30. <name>oidc.discoveryUri</name>
> 31. <value>https://login.microsoftonline.com/f82969***********1d0557a/.well-known/openid-configuration</value>
> 32. </param>
> 33. </provider>
> 34. <provider>
> 35. <role>identity-assertion</role>
> 36. <name>Default</name>
> 37. <enabled>true</enabled>
> 38. </provider>
> 39. </gateway>
> 40. <application>
> 41. <name>knoxauth</name>
> 42. </application>
> 43. <service>
> 44. <role>KNOXSSO</role>
> 45. <param>
> 46. <name>knoxsso.cookie.secure.only</name>
> 47. <value>false</value>
> 48. </param>
> 49. <param>
> 50. <name>knoxsso.token.ttl</name>
> 51. <value>30000</value>
> 52. </param>
> 53. <param>
> 54. <name>knoxsso.redirect.whitelist.regex</name>
> 55. <value>^https?:\/\/(dap-e0|x\.x\.2\.3|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$/value>
> 56. </param>
> 57. </service>
> 58. </topology>
>
> I tried using response_type "id_token", enabling nonces, knoxsso.secure to
> true, preferredJwsAlgorithm as RS256 etc. Nothing helps.
>
> gateway-audit.log when redirection error starts:
>
>
> 1. 18/02/15 12:38:02 ||7a66725e-6d9d-4ef5-9017-2b52d7d15ccf|audit|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?code=AQABAAIAAABHh4kmS_a**********************_WuZRkgVKneLpp83HnSlcntEbAmAgAA&state=0n7h1Y2LTz_**************99P92pZonRN-c&session_state=f0ac55a1-4***********-53e3e126b40e|success|Response status: 302
>
> It clearly shows Response status as "302" and not "200". This leads to
> redirection!
>
> What could I be missing here? Any pointers will be greatly appreciated.
> Regards
> Nisha
>
>