[Curtis Ruck <cu...@gmail.com>]

I'm trying to setup OIDC authentication, but with Nifi service existing
behind a reverse proxy, and for our other apps we use SSL Client
Authentication between reverse proxy and application, Nifi is picking up
the Reverse Proxy's SSL Certificate and falling into X509 Authentication
instead of OIDC. Any idea how I can disable X509 authentication in Nifi?

Connecting directly to nifi, it triggers the proper OIDC redirects.

--
Curtis Ruck

Re:

[Joe Witt <jo...@gmail.com>]

Curtis

Now that there is also a PR for this I'll comment directly there as
well to the specifics of the PR.

In reviewing the discussion here..

There is consensus that enabling the pattern of REST API interaction
you need for your case is a valuable capability.

However, we have not achieved consensus on how best to address it.

And in reviewing the PR and considering its impacts more broadly there
are concerns that it breaks the necessary model for site-to-site to
work properly and cluster replication.  These are designed to expect
certificates to be present and while proxying mechanisms are supported
through various headers the style you'd need in your model would not
be supported.

I dont believe your earlier statement that this change doesn't break
anything can be supported at this time. I'll comment to the specifics
of that concern on the PR so we keep PR.

If there is indeed a path to enable your use case and support existing
capabilities then that's great and lets find it.  Your expertise in
these proxies and authentication models combined with the community
knowledge of how NiFi works today and how to get it where it needs to
be is key.

Thanks
Joe
On Fri, Aug 10, 2018 at 8:59 AM Curtis Ruck <cu...@gmail.com> wrote:
>
> I created NIFI-5506 for the wantClientAuth specific issue, and submitted a WIP PR#2944 for review.
>
> Besides issues with getting OIDC working (on the OIDC Server side), this enables external providers.  Potentially, this could be amended to include X509 through reverse proxy by way of a request header, but considering that wouldn't work with a reverse proxy without this PR, I considered it out of scope of my near term issue.
>
> --
> Curtis Ruck
>
>
> On Thu, Aug 9, 2018 at 3:47 PM Curtis Ruck <cu...@gmail.com> wrote:
>>
>> The issue with Reverse Proxies and "certificates or other provider" is that if want=true, then the reverse proxy provides it's certificate, if a machine certificate is configured.  In Apache HTTPD, this machine certificate can be set at a Server or VHost level, not individual proxy rules, so to remove it for NiFi, i have to remove it for our other apps that "require" X509 client auth, and then either do a SSO workflow, or consume Reverse Proxy provided "authentication" details.  This turns "certificates or other provider" ends up at "reverse proxy certificate only".  So if Bob and Tim visit the reverse proxy, Nifi believes they are "Reverse Proxy" not Bob or Tim.
>>
>> Ideally I need "other provider", because my "other provider" does PKI authentication as part of SSO.  I could use "certificates or other provider", if NiFi could recognize Reverse Proxy validated certificates passed in via a request header.  If the reverse proxy doesn't provide a certificate, then it uses "other provider".  JBoss, Tomcat, all provide this functionality.
>>
>> I've already changed " else { setWantClientAuth(false); }" change in a fork and it got me closer to my customer's end goals.  I'm trying to get this change into NiFi so we don't have to maintain this fork.  I believe implementing this with a default want=true would not break any existing users of NiFi, and it would allow better integration with Reverse Proxies and Single Sign On.
>>
>> So In the near term i'd like a new nifi property setting to disable wantClientAuth, with the default enabled.
>> In the long term it would be ideal to support external authn/z providers as first class.
>>
>> --
>> My perspective comes from implementing Single Sign On in applications that don't always support it for over a decade for ~100 applications all sitting behind a Reverse Proxies, providing true single sign on without users having to do any special instructions to authenticate.  I'm a true believer that the best security is when the security doesn't impact the users, and proper single sign on allows application developers focus on their application's logic and not their AuthN/AuthZ security model.
>>
>> --
>> Curtis Ruck
>>
>>
>>
>> On Thu, Aug 9, 2018 at 3:00 PM Andy LoPresto <al...@apache.org> wrote:
>>>
>>> I think we agree in our assessment of what the code is doing and disagree in our desire for how that should occur. If OIDC is enabled and isClientAuthRequiredForRestApi() returns false, the result is:
>>>
>>> // Functionally equivalent to contextFactory.setNeedClientAuth(false);
>>> contextFactory.setWantClientAuth(true);
>>>
>>> That means that the server will request a client certificate if available, but will not require its presence to negotiate the TLS handshake. You are asking to set contextFactory.setWantClientAuth(false); as well, which will suppress the certificate selection dialog. If needClientAuth and wantClientAuth are both false, client certificates cannot be used to authenticate as they will never be sent from the browser. This will effectively allow you to choose between “certificates only”, “certificates or other provider”, and “other provider only (no certificates)”.
>>>
>>> I am saying that core NiFi *always* accepts client certificates as an authentication mechanism; there is no scenario in which need and want are both set to false. This is by design. Again, I am not saying this can never change, but because of the expectations, documentation, and shared knowledge around this mechanism, changing it is (in my opinion) a major change, and should not be done in a minor release. Other project members may (and probably do) disagree with me.
>>>
>>> A property in nifi.properties which defaults to “off” but when manually enabled can bypass this requirement is an option. I don’t think we disagree on how to implement this specific change; I think we differ only on how impactful it will be. My perspective comes from supporting a large number of users with a broad variety of (often conflicting) requirements, and sometimes (both they and I have) very little knowledge of the rest of their IT ecosystem. I believe your perspective comes from a specific user with specific requirements. That’s why I recommended making the localized change you need in a fork of the project, so you can achieve your objective in a timeframe that is not blocked by other parties.
>>>
>>> Andy LoPresto
>>> alopresto@apache.org
>>> alopresto.apache@gmail.com
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Aug 9, 2018, at 11:48 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>>
>>> In my environment, by trying to enable OIDC, it returns false in that function you selected.
>>>
>>> My suggestion, is that in the } else { block, changing the setWantClientAuth(true) to setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to true in the absence of the setting.
>>>
>>> By allowing a property to disable this check, would neuter the current X509 Authentication, as it won't have a certificate to authenticate.  It would also address Shawn's concern of for having his users cancel the first authentication popup.
>>>
>>> It's not fixing AuthN/AuthZ, but its allowing us in special circumstances to disable X509 easily.  In my environment, it's even preferable because we would use OIDC to redirect to Apereo CAS, which does X509 Authentication itself.
>>>
>>> --
>>> Curtis Ruck
>>>
>>>
>>> On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <al...@apache.org> wrote:
>>>>
>>>> Hi Curtis,
>>>>
>>>> There has definitely been some discussion about this and it is picking up recently. I understand the difficulty faced when using NiFi in conjunction with a reverse proxy or external identity provider. I am not saying the current way is perfect, or even the best.
>>>>
>>>> …but…
>>>>
>>>> Always allowing X.509 authentication has been the standard for years (due to NiFi’s original design requirements), and changing this will have far-reaching impact on the application. The additional identity providers have been added piece-meal as requirements arose, and because of the multiple contexts NiFi supports, there are additional changes required for each — issuing one time tokens for file downloads, etc.
>>>>
>>>> The code you identified returns a boolean value, but it is not simply reading a boolean flag from the properties file — it is calculating the presence/absence of the other (non-credential-based) identity providers:
>>>>
>>>> /**
>>>>  * Returns true if client certificates are required for REST API. Determined
>>>>  * if the following conditions are all true:
>>>>  * <p>
>>>>  * - login identity provider is not populated
>>>>  * - Kerberos service support is not enabled
>>>>  * - openid connect is not enabled
>>>>  * - knox sso is not enabled
>>>>  * </p>
>>>>  *
>>>>  * @return true if client certificates are required for access to the REST API
>>>>  */
>>>> public boolean isClientAuthRequiredForRestApi() {
>>>>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
>>>> }
>>>>
>>>> There is active planning for a complete rewrite of the authentication mechanism ordering, but because this is such a wide-reaching change and will have substantial impact across the project, I strongly advocate for this to go in a major release. As the number of users of the project continues to grow, we have to balance improving the experience in edge scenarios against more common scenarios. While I don’t mean to negate the times a reverse proxy is required, it is not present in the majority of deployments, and the current model works sufficiently in those deployments. We have to keep those deployments in mind as well while we make these changes.
>>>>
>>>> If you have an immediate need to deploy NiFi behind a reverse proxy and disable the X.509 requests, my honest suggestion would be to fork the repository and apply a patch to return false from that method just as you specified. Unfortunately, that patch alone probably would not be accepted into the upstream NiFi project at this time. It can certainly be documented as a requirement for the rearchitected authentication mechanism moving forward.
>>>>
>>>> Thanks for sharing your expectations and needs from the project, and hopefully we can meet them soon.
>>>>
>>>>
>>>> Andy LoPresto
>>>> alopresto@apache.org
>>>> alopresto.apache@gmail.com
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>>>
>>>> FYSA,
>>>>
>>>> This is where X509 is "always-on".
>>>>
>>>> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>>>>
>>>> if (props.isClientAuthRequiredForRestApi()) {
>>>>    contextFactory.setNeedClientAuth(true);
>>>> } else {
>>>>    contextFactory.setWantClientAuth(true);
>>>> }
>>>>
>>>> I believe in the short term, modifying this section to use nifi.properties to allow us to provide a false to wantClientAuth, would address our concerns.
>>>>
>>>> --
>>>> Curtis Ruck
>>>>
>>>>
>>>> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <cu...@gmail.com> wrote:
>>>>>
>>>>> To support Shawn's statement even further.  If my customer can't get NiFi to operate behind our reverse proxy, it won't be in our system.  I'm trying to find the easiest approach, and NiFi's OIDC should be perfect, if X509 wasn't "wanted" up front.
>>>>>
>>>>> I'd argue that all of the AuthN/AuthZ code should be abstracted out significantly more than it currently is, with the ability to completely configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>>>>> The ability to manage users/groups in NiFi's UI should be a toggle.
>>>>> There should be an easy higher level API to use for group/role provisioning.  If a new user "bob" open's NiFi and they have a "read-only" role, then they shouldn't need to be manually provisioned in NiFi, and we my customer tries to minimize the number of unique applications reaching into LDAP.  Every application that implements LDAP support implements it differently, and they don't always scale up appropriately.
>>>>>
>>>>> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.  With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's custom protocols, which can support Yubikey, Google Authentication, ADFS, Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any of it.
>>>>>
>>>>> I'm even willing to assist in providing some PRs to move NiFi in the right direction, I just think we should figure out the higher level architecture/design a little better; especially since NiFi's job is to help things integrate together, it's not being a good team player.
>>>>>
>>>>> As much as I hate to say it, if NiFi was a proper Java EE project, I could just use a war overlay to modify the AuthN/AuthZ to success; even if it was just a self-executing .war.
>>>>>
>>>>> --
>>>>> Curtis Ruck
>>>>>
>>>>>
>>>>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us> wrote:
>>>>>>
>>>>>> I'll clarify my statement a little as well with a workflow.
>>>>>>
>>>>>> You open the NiFi UI Link
>>>>>> Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>>>>> Then you get Prompts for Username and Password because of GSSAPI even though your not on that REALM.
>>>>>> Then you get directed to the Identify Management Reverse Proxy URL for Knox SSO
>>>>>> Then you get prompted for your Certificate which you should select.
>>>>>> Then you might get prompted for Kerberos Again which you cancel
>>>>>> Finally your in NiFi.
>>>>>>
>>>>>>
>>>>>> Painful doesn't even begin to describe it lol.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Shawn
>>>>>> ________________________________
>>>>>> From: Kevin Doran <kd...@apache.org>
>>>>>> Sent: Thursday, August 9, 2018 11:07:28 AM
>>>>>> To: users@nifi.apache.org
>>>>>> Subject: Re: Re:
>>>>>>
>>>>>>
>>>>>> Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful
>>>>>>
>>>>>>
>>>>>> Wow, that sounds terrible. Confusing, accident prone, and frustrating to correct mistakes (at least in my experience, forcing a browser to forget client certificate preferences is difficult).
>>>>>>
>>>>>> Thanks for sharing those details about your deployment scenario. This can definitely be improved and I have some ideas for how to do it. I've cloned the issue to NiFi to make sure we are tracking it for both projects [1][2]
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/NIFIREG-189
>>>>>> [2] https://issues.apache.org/jira/browse/NIFI-5504
>>>>>>
>>>>>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us> wrote:
>>>>>>
>>>>>> The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.
>>>>>>
>>>>>> We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.
>>>>>>
>>>>>> Thanks
>>>>>> Shawnk
>>>>>> ________________________________
>>>>>> From: Kevin Doran <kd...@apache.org>
>>>>>> Sent: Thursday, August 9, 2018 10:47:56 AM
>>>>>> To: users@nifi.apache.org
>>>>>> Subject: Re:
>>>>>>
>>>>>> sorry forgot the link. here it is:
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>>>>>>
>>>>>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>>>>>>
>>>>>> Hi Curtis,
>>>>>>
>>>>>> This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.
>>>>>>
>>>>>> Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?
>>>>>>
>>>>>> Thanks,
>>>>>> Kevin
>>>>>>
>>>>>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>>>>>
>>>>>> I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?
>>>>>>
>>>>>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>>>>>
>>>>>> --
>>>>>> Curtis Ruck
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>

Re:

[Curtis Ruck <cu...@gmail.com>]

I created NIFI-5506 for the wantClientAuth specific issue, and submitted a
WIP PR#2944 for review.

Besides issues with getting OIDC working (on the OIDC Server side), this
enables external providers.  Potentially, this could be amended to include
X509 through reverse proxy by way of a request header, but considering that
wouldn't work with a reverse proxy without this PR, I considered it out of
scope of my near term issue.

--
Curtis Ruck


On Thu, Aug 9, 2018 at 3:47 PM Curtis Ruck <cu...@gmail.com> wrote:

> The issue with Reverse Proxies and "certificates or other provider" is
> that if want=true, then the reverse proxy provides it's certificate, if a
> machine certificate is configured.  In Apache HTTPD, this machine
> certificate can be set at a Server or VHost level, not individual proxy
> rules, so to remove it for NiFi, i have to remove it for our other apps
> that "require" X509 client auth, and then either do a SSO workflow, or
> consume Reverse Proxy provided "authentication" details.  This turns
> "certificates or other provider" ends up at "reverse proxy certificate
> only".  So if Bob and Tim visit the reverse proxy, Nifi believes they are
> "Reverse Proxy" not Bob or Tim.
>
> Ideally I need "other provider", because my "other provider" does PKI
> authentication as part of SSO.  I could use "certificates or other
> provider", if NiFi could recognize Reverse Proxy validated certificates
> passed in via a request header.  If the reverse proxy doesn't provide a
> certificate, then it uses "other provider".  JBoss, Tomcat, all provide
> this functionality.
>
> I've already changed " else { setWantClientAuth(false); }" change in a
> fork and it got me closer to my customer's end goals.  I'm trying to get
> this change into NiFi so we don't have to maintain this fork.  I believe
> implementing this with a default want=true would not break any existing
> users of NiFi, and it would allow better integration with Reverse Proxies
> and Single Sign On.
>
> So In the near term i'd like a new nifi property setting to disable
> wantClientAuth, with the default enabled.
> In the long term it would be ideal to support external authn/z providers
> as first class.
>
> --
> My perspective comes from implementing Single Sign On in applications that
> don't always support it for over a decade for ~100 applications all sitting
> behind a Reverse Proxies, providing true single sign on without users
> having to do any special instructions to authenticate.  I'm a true believer
> that the best security is when the security doesn't impact the users, and
> proper single sign on allows application developers focus on their
> application's logic and not their AuthN/AuthZ security model.
>
> --
> Curtis Ruck
>
>
>
> On Thu, Aug 9, 2018 at 3:00 PM Andy LoPresto <al...@apache.org> wrote:
>
>> I think we agree in our assessment of what the code is doing and disagree
>> in our desire for how that should occur. If OIDC is enabled and
>> isClientAuthRequiredForRestApi() returns false, the result is:
>>
>> // Functionally equivalent to contextFactory.setNeedClientAuth(false);
>> contextFactory.setWantClientAuth(true);
>>
>> That means that the server will request a client certificate if
>> available, but will not require its presence to negotiate the TLS
>> handshake. You are asking to set contextFactory.setWantClientAuth(false);
>> as well, which will suppress the certificate selection dialog. If
>> needClientAuth and wantClientAuth are both false, client certificates
>> cannot be used to authenticate as they will never be sent from the browser.
>> This will effectively allow you to choose between “certificates only”,
>> “certificates or other provider”, and “other provider only (no
>> certificates)”.
>>
>> I am saying that core NiFi *always* accepts client certificates as an
>> authentication mechanism; there is no scenario in which need and want are
>> both set to false. This is by design. Again, I am not saying this can never
>> change, but because of the expectations, documentation, and shared
>> knowledge around this mechanism, changing it is (in my opinion) a major
>> change, and should not be done in a minor release. Other project members
>> may (and probably do) disagree with me.
>>
>> A property in nifi.properties which defaults to “off” but when manually
>> enabled can bypass this requirement is an option. I don’t think we disagree
>> on how to implement this specific change; I think we differ only on how
>> impactful it will be. My perspective comes from supporting a large number
>> of users with a broad variety of (often conflicting) requirements, and
>> sometimes (both they and I have) very little knowledge of the rest of their
>> IT ecosystem. I believe your perspective comes from a specific user with
>> specific requirements. That’s why I recommended making the localized change
>> you need in a fork of the project, so you can achieve your objective in a
>> timeframe that is not blocked by other parties.
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <al...@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Aug 9, 2018, at 11:48 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>
>> In my environment, by trying to enable OIDC, it returns false in that
>> function you selected.
>>
>> My suggestion, is that in the } else { block, changing the
>> setWantClientAuth(true) to
>> setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to
>> true in the absence of the setting.
>>
>> By allowing a property to disable this check, would neuter the current
>> X509 Authentication, as it won't have a certificate to authenticate.  It
>> would also address Shawn's concern of for having his users cancel the first
>> authentication popup.
>>
>> It's not fixing AuthN/AuthZ, but its allowing us in special circumstances
>> to disable X509 easily.  In my environment, it's even preferable because we
>> would use OIDC to redirect to Apereo CAS, which does X509 Authentication
>> itself.
>>
>> --
>> Curtis Ruck
>>
>>
>> On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <al...@apache.org>
>> wrote:
>>
>>> Hi Curtis,
>>>
>>> There has definitely been some discussion about this and it is picking
>>> up recently. I understand the difficulty faced when using NiFi in
>>> conjunction with a reverse proxy or external identity provider. I am not
>>> saying the current way is perfect, or even the best.
>>>
>>> …but…
>>>
>>> Always allowing X.509 authentication has been the standard for years
>>> (due to NiFi’s original design requirements), and changing this will have
>>> far-reaching impact on the application. The additional identity providers
>>> have been added piece-meal as requirements arose, and because of the
>>> multiple contexts NiFi supports, there are additional changes required for
>>> each — issuing one time tokens for file downloads, etc.
>>>
>>> The code you identified returns a boolean value, but it is not simply
>>> reading a boolean flag from the properties file — it is calculating the
>>> presence/absence of the other (non-credential-based) identity providers:
>>>
>>> /**
>>>  * Returns true if client certificates are required for REST API. Determined
>>>  * if the following conditions are all true:
>>>  * <p>
>>>  * - login identity provider is not populated
>>>  * - Kerberos service support is not enabled
>>>  * - openid connect is not enabled
>>>  * - knox sso is not enabled
>>>  * </p>
>>>  *
>>>  * @return true if client certificates are required for access to the REST API
>>>  */
>>> public boolean isClientAuthRequiredForRestApi() {
>>>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
>>> }
>>>
>>> There is active planning for a complete rewrite of the authentication
>>> mechanism ordering, but because this is such a wide-reaching change and
>>> will have substantial impact across the project, I strongly advocate for
>>> this to go in a major release. As the number of users of the project
>>> continues to grow, we have to balance improving the experience in edge
>>> scenarios against more common scenarios. While I don’t mean to negate the
>>> times a reverse proxy is required, it is not present in the majority of
>>> deployments, and the current model works sufficiently in those deployments.
>>> We have to keep those deployments in mind as well while we make these
>>> changes.
>>>
>>> If you have an immediate need to deploy NiFi behind a reverse proxy and
>>> disable the X.509 requests, my honest suggestion would be to fork the
>>> repository and apply a patch to return false from that method just as you
>>> specified. Unfortunately, that patch alone probably would not be accepted
>>> into the upstream NiFi project at this time. It can certainly be documented
>>> as a requirement for the rearchitected authentication mechanism moving
>>> forward.
>>>
>>> Thanks for sharing your expectations and needs from the project, and
>>> hopefully we can meet them soon.
>>>
>>>
>>> Andy LoPresto
>>> alopresto@apache.org
>>> *alopresto.apache@gmail.com <al...@gmail.com>*
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>>
>>> FYSA,
>>>
>>> This is where X509 is "always-on".
>>>
>>>
>>> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>>>
>>> if (props.isClientAuthRequiredForRestApi()) {
>>>    contextFactory.setNeedClientAuth(true);
>>> } else {
>>>    contextFactory.setWantClientAuth(true);
>>> }
>>>
>>> I believe in the short term, modifying this section to use
>>> nifi.properties to allow us to provide a false to wantClientAuth, would
>>> address our concerns.
>>>
>>> --
>>> Curtis Ruck
>>>
>>>
>>> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <cu...@gmail.com>
>>> wrote:
>>>
>>>> To support Shawn's statement even further.  If my customer can't get
>>>> NiFi to operate behind our reverse proxy, it won't be in our system.  I'm
>>>> trying to find the easiest approach, and NiFi's OIDC should be perfect, if
>>>> X509 wasn't "wanted" up front.
>>>>
>>>> I'd argue that all of the AuthN/AuthZ code should be abstracted out
>>>> significantly more than it currently is, with the ability to completely
>>>> configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>>>> The ability to manage users/groups in NiFi's UI should be a toggle.
>>>> There should be an easy higher level API to use for group/role
>>>> provisioning.  If a new user "bob" open's NiFi and they have a "read-only"
>>>> role, then they shouldn't need to be manually provisioned in NiFi, and we
>>>> my customer tries to minimize the number of unique applications reaching
>>>> into LDAP.  Every application that implements LDAP support implements it
>>>> differently, and they don't always scale up appropriately.
>>>>
>>>> For example, i'm trying to get Apereo CAS 5.x working with Apache
>>>> NiFi.  With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or
>>>> CAS's custom protocols, which can support Yubikey, Google Authentication,
>>>> ADFS, Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't
>>>> use any of it.
>>>>
>>>> I'm even willing to assist in providing some PRs to move NiFi in the
>>>> right direction, I just think we should figure out the higher level
>>>> architecture/design a little better; especially since NiFi's job is to help
>>>> things integrate together, it's not being a good team player.
>>>>
>>>> As much as I hate to say it, if NiFi was a proper Java EE project, I
>>>> could just use a war overlay to modify the AuthN/AuthZ to success; even if
>>>> it was just a self-executing .war.
>>>>
>>>> --
>>>> Curtis Ruck
>>>>
>>>>
>>>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us>
>>>> wrote:
>>>>
>>>>> I'll clarify my statement a little as well with a workflow.
>>>>>
>>>>>
>>>>>    1. You open the NiFi UI Link
>>>>>    2. Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>>>>    3. Then you get Prompts for Username and Password because of
>>>>>    GSSAPI even though your not on that REALM.
>>>>>    4. Then you get directed to the Identify Management Reverse Proxy
>>>>>    URL for Knox SSO
>>>>>    5. Then you get prompted for your Certificate which you should
>>>>>    select.
>>>>>    6. Then you might get prompted for Kerberos Again which you cancel
>>>>>    7. Finally your in NiFi.
>>>>>
>>>>>
>>>>> Painful doesn't even begin to describe it lol.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Shawn
>>>>> ------------------------------
>>>>> *From:* Kevin Doran <kd...@apache.org>
>>>>> *Sent:* Thursday, August 9, 2018 11:07:28 AM
>>>>> *To:* users@nifi.apache.org
>>>>> *Subject:* Re: Re:
>>>>>
>>>>>
>>>>> *Explaining to your end users that you should skip the first
>>>>> Certificate Prompt but accept the second but only when you haven't logged
>>>>> in the current session is really painful*
>>>>>
>>>>>
>>>>> Wow, that sounds terrible. Confusing, accident prone, and frustrating
>>>>> to correct mistakes (at least in my experience, forcing a browser to forget
>>>>> client certificate preferences is difficult).
>>>>>
>>>>> Thanks for sharing those details about your deployment scenario. This
>>>>> can definitely be improved and I have some ideas for how to do it. I've
>>>>> cloned the issue to NiFi to make sure we are tracking it for both projects
>>>>> [1][2]
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/NIFIREG-189
>>>>> [2] https://issues.apache.org/jira/browse/NIFI-5504
>>>>>
>>>>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <
>>>>> sweeks@weeksconsulting.us> wrote:
>>>>>
>>>>> The project I'm on is running into this issue as well and it gets
>>>>> particularly painful when all of your server's are signed by the same root
>>>>> ca that signs your smart card logins and your using something like KnoxSSO.
>>>>> Explaining to your end users that you should skip the first Certificate
>>>>> Prompt but accept the second but only when you haven't logged in the
>>>>> current session is really painful and shows major shortcoming between the
>>>>> back end authentication between servers and front end ui authentication.
>>>>>
>>>>> We can't even considering putting it behind our identify reverse
>>>>> proxies because we can't turn off two way ssl.
>>>>>
>>>>> Thanks
>>>>> Shawnk
>>>>> ------------------------------
>>>>> *From:* Kevin Doran <kd...@apache.org>
>>>>> *Sent:* Thursday, August 9, 2018 10:47:56 AM
>>>>> *To:* users@nifi.apache.org
>>>>> *Subject:* Re:
>>>>>
>>>>> sorry forgot the link. here it is:
>>>>>
>>>>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>>>>>
>>>>>
>>>>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> This has come up a few times. Unfortunately I don’t think there is
>>>>> currently an easy way to disable X509-based identity extraction in NiFi
>>>>> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
>>>>> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
>>>>> NiFi, so this ticket should apply to NiFi as well.
>>>>>
>>>>> Perhaps you could share more about your needs and use case on that
>>>>> ticket so that when it gets implemented we could take that scenario with
>>>>> reverse proxies and OIDC into account?
>>>>>
>>>>> Thanks,
>>>>> Kevin
>>>>>
>>>>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> I'm trying to setup OIDC authentication, but with Nifi service
>>>>> existing behind a reverse proxy, and for our other apps we use SSL Client
>>>>> Authentication between reverse proxy and application, Nifi is picking up
>>>>> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
>>>>> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>>>>>
>>>>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>>>>
>>>>> --
>>>>> Curtis Ruck
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>

Re:

[Curtis Ruck <cu...@gmail.com>]

The issue with Reverse Proxies and "certificates or other provider" is that
if want=true, then the reverse proxy provides it's certificate, if a
machine certificate is configured.  In Apache HTTPD, this machine
certificate can be set at a Server or VHost level, not individual proxy
rules, so to remove it for NiFi, i have to remove it for our other apps
that "require" X509 client auth, and then either do a SSO workflow, or
consume Reverse Proxy provided "authentication" details.  This turns
"certificates or other provider" ends up at "reverse proxy certificate
only".  So if Bob and Tim visit the reverse proxy, Nifi believes they are
"Reverse Proxy" not Bob or Tim.

Ideally I need "other provider", because my "other provider" does PKI
authentication as part of SSO.  I could use "certificates or other
provider", if NiFi could recognize Reverse Proxy validated certificates
passed in via a request header.  If the reverse proxy doesn't provide a
certificate, then it uses "other provider".  JBoss, Tomcat, all provide
this functionality.

I've already changed " else { setWantClientAuth(false); }" change in a fork
and it got me closer to my customer's end goals.  I'm trying to get this
change into NiFi so we don't have to maintain this fork.  I believe
implementing this with a default want=true would not break any existing
users of NiFi, and it would allow better integration with Reverse Proxies
and Single Sign On.

So In the near term i'd like a new nifi property setting to disable
wantClientAuth, with the default enabled.
In the long term it would be ideal to support external authn/z providers as
first class.

--
My perspective comes from implementing Single Sign On in applications that
don't always support it for over a decade for ~100 applications all sitting
behind a Reverse Proxies, providing true single sign on without users
having to do any special instructions to authenticate.  I'm a true believer
that the best security is when the security doesn't impact the users, and
proper single sign on allows application developers focus on their
application's logic and not their AuthN/AuthZ security model.

--
Curtis Ruck



On Thu, Aug 9, 2018 at 3:00 PM Andy LoPresto <al...@apache.org> wrote:

> I think we agree in our assessment of what the code is doing and disagree
> in our desire for how that should occur. If OIDC is enabled and
> isClientAuthRequiredForRestApi() returns false, the result is:
>
> // Functionally equivalent to contextFactory.setNeedClientAuth(false);
> contextFactory.setWantClientAuth(true);
>
> That means that the server will request a client certificate if available,
> but will not require its presence to negotiate the TLS handshake. You are
> asking to set contextFactory.setWantClientAuth(false); as well, which will
> suppress the certificate selection dialog. If needClientAuth and
> wantClientAuth are both false, client certificates cannot be used to
> authenticate as they will never be sent from the browser. This will
> effectively allow you to choose between “certificates only”, “certificates
> or other provider”, and “other provider only (no certificates)”.
>
> I am saying that core NiFi *always* accepts client certificates as an
> authentication mechanism; there is no scenario in which need and want are
> both set to false. This is by design. Again, I am not saying this can never
> change, but because of the expectations, documentation, and shared
> knowledge around this mechanism, changing it is (in my opinion) a major
> change, and should not be done in a minor release. Other project members
> may (and probably do) disagree with me.
>
> A property in nifi.properties which defaults to “off” but when manually
> enabled can bypass this requirement is an option. I don’t think we disagree
> on how to implement this specific change; I think we differ only on how
> impactful it will be. My perspective comes from supporting a large number
> of users with a broad variety of (often conflicting) requirements, and
> sometimes (both they and I have) very little knowledge of the rest of their
> IT ecosystem. I believe your perspective comes from a specific user with
> specific requirements. That’s why I recommended making the localized change
> you need in a fork of the project, so you can achieve your objective in a
> timeframe that is not blocked by other parties.
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Aug 9, 2018, at 11:48 AM, Curtis Ruck <cu...@gmail.com> wrote:
>
> In my environment, by trying to enable OIDC, it returns false in that
> function you selected.
>
> My suggestion, is that in the } else { block, changing the
> setWantClientAuth(true) to
> setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to
> true in the absence of the setting.
>
> By allowing a property to disable this check, would neuter the current
> X509 Authentication, as it won't have a certificate to authenticate.  It
> would also address Shawn's concern of for having his users cancel the first
> authentication popup.
>
> It's not fixing AuthN/AuthZ, but its allowing us in special circumstances
> to disable X509 easily.  In my environment, it's even preferable because we
> would use OIDC to redirect to Apereo CAS, which does X509 Authentication
> itself.
>
> --
> Curtis Ruck
>
>
> On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <al...@apache.org> wrote:
>
>> Hi Curtis,
>>
>> There has definitely been some discussion about this and it is picking up
>> recently. I understand the difficulty faced when using NiFi in conjunction
>> with a reverse proxy or external identity provider. I am not saying the
>> current way is perfect, or even the best.
>>
>> …but…
>>
>> Always allowing X.509 authentication has been the standard for years (due
>> to NiFi’s original design requirements), and changing this will have
>> far-reaching impact on the application. The additional identity providers
>> have been added piece-meal as requirements arose, and because of the
>> multiple contexts NiFi supports, there are additional changes required for
>> each — issuing one time tokens for file downloads, etc.
>>
>> The code you identified returns a boolean value, but it is not simply
>> reading a boolean flag from the properties file — it is calculating the
>> presence/absence of the other (non-credential-based) identity providers:
>>
>> /**
>>  * Returns true if client certificates are required for REST API. Determined
>>  * if the following conditions are all true:
>>  * <p>
>>  * - login identity provider is not populated
>>  * - Kerberos service support is not enabled
>>  * - openid connect is not enabled
>>  * - knox sso is not enabled
>>  * </p>
>>  *
>>  * @return true if client certificates are required for access to the REST API
>>  */
>> public boolean isClientAuthRequiredForRestApi() {
>>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
>> }
>>
>> There is active planning for a complete rewrite of the authentication
>> mechanism ordering, but because this is such a wide-reaching change and
>> will have substantial impact across the project, I strongly advocate for
>> this to go in a major release. As the number of users of the project
>> continues to grow, we have to balance improving the experience in edge
>> scenarios against more common scenarios. While I don’t mean to negate the
>> times a reverse proxy is required, it is not present in the majority of
>> deployments, and the current model works sufficiently in those deployments.
>> We have to keep those deployments in mind as well while we make these
>> changes.
>>
>> If you have an immediate need to deploy NiFi behind a reverse proxy and
>> disable the X.509 requests, my honest suggestion would be to fork the
>> repository and apply a patch to return false from that method just as you
>> specified. Unfortunately, that patch alone probably would not be accepted
>> into the upstream NiFi project at this time. It can certainly be documented
>> as a requirement for the rearchitected authentication mechanism moving
>> forward.
>>
>> Thanks for sharing your expectations and needs from the project, and
>> hopefully we can meet them soon.
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <al...@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <cu...@gmail.com> wrote:
>>
>> FYSA,
>>
>> This is where X509 is "always-on".
>>
>>
>> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>>
>> if (props.isClientAuthRequiredForRestApi()) {
>>    contextFactory.setNeedClientAuth(true);
>> } else {
>>    contextFactory.setWantClientAuth(true);
>> }
>>
>> I believe in the short term, modifying this section to use
>> nifi.properties to allow us to provide a false to wantClientAuth, would
>> address our concerns.
>>
>> --
>> Curtis Ruck
>>
>>
>> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <cu...@gmail.com>
>> wrote:
>>
>>> To support Shawn's statement even further.  If my customer can't get
>>> NiFi to operate behind our reverse proxy, it won't be in our system.  I'm
>>> trying to find the easiest approach, and NiFi's OIDC should be perfect, if
>>> X509 wasn't "wanted" up front.
>>>
>>> I'd argue that all of the AuthN/AuthZ code should be abstracted out
>>> significantly more than it currently is, with the ability to completely
>>> configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>>> The ability to manage users/groups in NiFi's UI should be a toggle.
>>> There should be an easy higher level API to use for group/role
>>> provisioning.  If a new user "bob" open's NiFi and they have a "read-only"
>>> role, then they shouldn't need to be manually provisioned in NiFi, and we
>>> my customer tries to minimize the number of unique applications reaching
>>> into LDAP.  Every application that implements LDAP support implements it
>>> differently, and they don't always scale up appropriately.
>>>
>>> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.
>>> With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's
>>> custom protocols, which can support Yubikey, Google Authentication, ADFS,
>>> Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any
>>> of it.
>>>
>>> I'm even willing to assist in providing some PRs to move NiFi in the
>>> right direction, I just think we should figure out the higher level
>>> architecture/design a little better; especially since NiFi's job is to help
>>> things integrate together, it's not being a good team player.
>>>
>>> As much as I hate to say it, if NiFi was a proper Java EE project, I
>>> could just use a war overlay to modify the AuthN/AuthZ to success; even if
>>> it was just a self-executing .war.
>>>
>>> --
>>> Curtis Ruck
>>>
>>>
>>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us>
>>> wrote:
>>>
>>>> I'll clarify my statement a little as well with a workflow.
>>>>
>>>>
>>>>    1. You open the NiFi UI Link
>>>>    2. Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>>>    3. Then you get Prompts for Username and Password because of GSSAPI
>>>>    even though your not on that REALM.
>>>>    4. Then you get directed to the Identify Management Reverse Proxy
>>>>    URL for Knox SSO
>>>>    5. Then you get prompted for your Certificate which you should
>>>>    select.
>>>>    6. Then you might get prompted for Kerberos Again which you cancel
>>>>    7. Finally your in NiFi.
>>>>
>>>>
>>>> Painful doesn't even begin to describe it lol.
>>>>
>>>>
>>>> Thanks
>>>> Shawn
>>>> ------------------------------
>>>> *From:* Kevin Doran <kd...@apache.org>
>>>> *Sent:* Thursday, August 9, 2018 11:07:28 AM
>>>> *To:* users@nifi.apache.org
>>>> *Subject:* Re: Re:
>>>>
>>>>
>>>> *Explaining to your end users that you should skip the first
>>>> Certificate Prompt but accept the second but only when you haven't logged
>>>> in the current session is really painful*
>>>>
>>>>
>>>> Wow, that sounds terrible. Confusing, accident prone, and frustrating
>>>> to correct mistakes (at least in my experience, forcing a browser to forget
>>>> client certificate preferences is difficult).
>>>>
>>>> Thanks for sharing those details about your deployment scenario. This
>>>> can definitely be improved and I have some ideas for how to do it. I've
>>>> cloned the issue to NiFi to make sure we are tracking it for both projects
>>>> [1][2]
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFIREG-189
>>>> [2] https://issues.apache.org/jira/browse/NIFI-5504
>>>>
>>>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sweeks@weeksconsulting.us
>>>> > wrote:
>>>>
>>>> The project I'm on is running into this issue as well and it gets
>>>> particularly painful when all of your server's are signed by the same root
>>>> ca that signs your smart card logins and your using something like KnoxSSO.
>>>> Explaining to your end users that you should skip the first Certificate
>>>> Prompt but accept the second but only when you haven't logged in the
>>>> current session is really painful and shows major shortcoming between the
>>>> back end authentication between servers and front end ui authentication.
>>>>
>>>> We can't even considering putting it behind our identify reverse
>>>> proxies because we can't turn off two way ssl.
>>>>
>>>> Thanks
>>>> Shawnk
>>>> ------------------------------
>>>> *From:* Kevin Doran <kd...@apache.org>
>>>> *Sent:* Thursday, August 9, 2018 10:47:56 AM
>>>> *To:* users@nifi.apache.org
>>>> *Subject:* Re:
>>>>
>>>> sorry forgot the link. here it is:
>>>>
>>>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>>>>
>>>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>>>>
>>>> Hi Curtis,
>>>>
>>>> This has come up a few times. Unfortunately I don’t think there is
>>>> currently an easy way to disable X509-based identity extraction in NiFi
>>>> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
>>>> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
>>>> NiFi, so this ticket should apply to NiFi as well.
>>>>
>>>> Perhaps you could share more about your needs and use case on that
>>>> ticket so that when it gets implemented we could take that scenario with
>>>> reverse proxies and OIDC into account?
>>>>
>>>> Thanks,
>>>> Kevin
>>>>
>>>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
>>>> wrote:
>>>>
>>>> I'm trying to setup OIDC authentication, but with Nifi service existing
>>>> behind a reverse proxy, and for our other apps we use SSL Client
>>>> Authentication between reverse proxy and application, Nifi is picking up
>>>> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
>>>> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>>>>
>>>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>>>
>>>> --
>>>> Curtis Ruck
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>

Re:

[Andy LoPresto <al...@apache.org>]

Also, please let me know if I am missing something or not accurately understanding your requirements/suggestion. I don’t want a situation where I’m arguing past you and not addressing what you need to be successful.

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 9, 2018, at 12:00 PM, Andy LoPresto <al...@apache.org> wrote:
> 
> I think we agree in our assessment of what the code is doing and disagree in our desire for how that should occur. If OIDC is enabled and isClientAuthRequiredForRestApi() returns false, the result is:
> 
> // Functionally equivalent to contextFactory.setNeedClientAuth(false);
> contextFactory.setWantClientAuth(true);
> 
> That means that the server will request a client certificate if available, but will not require its presence to negotiate the TLS handshake. You are asking to set contextFactory.setWantClientAuth(false); as well, which will suppress the certificate selection dialog. If needClientAuth and wantClientAuth are both false, client certificates cannot be used to authenticate as they will never be sent from the browser. This will effectively allow you to choose between “certificates only”, “certificates or other provider”, and “other provider only (no certificates)”.
> 
> I am saying that core NiFi *always* accepts client certificates as an authentication mechanism; there is no scenario in which need and want are both set to false. This is by design. Again, I am not saying this can never change, but because of the expectations, documentation, and shared knowledge around this mechanism, changing it is (in my opinion) a major change, and should not be done in a minor release. Other project members may (and probably do) disagree with me.
> 
> A property in nifi.properties which defaults to “off” but when manually enabled can bypass this requirement is an option. I don’t think we disagree on how to implement this specific change; I think we differ only on how impactful it will be. My perspective comes from supporting a large number of users with a broad variety of (often conflicting) requirements, and sometimes (both they and I have) very little knowledge of the rest of their IT ecosystem. I believe your perspective comes from a specific user with specific requirements. That’s why I recommended making the localized change you need in a fork of the project, so you can achieve your objective in a timeframe that is not blocked by other parties.
> 
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com <ma...@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Aug 9, 2018, at 11:48 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>> 
>> In my environment, by trying to enable OIDC, it returns false in that function you selected.
>> 
>> My suggestion, is that in the } else { block, changing the setWantClientAuth(true) to setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to true in the absence of the setting.
>> 
>> By allowing a property to disable this check, would neuter the current X509 Authentication, as it won't have a certificate to authenticate.  It would also address Shawn's concern of for having his users cancel the first authentication popup.
>> 
>> It's not fixing AuthN/AuthZ, but its allowing us in special circumstances to disable X509 easily.  In my environment, it's even preferable because we would use OIDC to redirect to Apereo CAS, which does X509 Authentication itself.
>> 
>> --
>> Curtis Ruck
>> 
>> 
>> On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <alopresto@apache.org <ma...@apache.org>> wrote:
>> Hi Curtis,
>> 
>> There has definitely been some discussion about this and it is picking up recently. I understand the difficulty faced when using NiFi in conjunction with a reverse proxy or external identity provider. I am not saying the current way is perfect, or even the best.
>> 
>> …but…
>> 
>> Always allowing X.509 authentication has been the standard for years (due to NiFi’s original design requirements), and changing this will have far-reaching impact on the application. The additional identity providers have been added piece-meal as requirements arose, and because of the multiple contexts NiFi supports, there are additional changes required for each — issuing one time tokens for file downloads, etc.
>> 
>> The code you identified returns a boolean value, but it is not simply reading a boolean flag from the properties file — it is calculating the presence/absence of the other (non-credential-based) identity providers:
>> 
>> /**
>>  * Returns true if client certificates are required for REST API. Determined
>>  * if the following conditions are all true:
>>  * <p>
>>  * - login identity provider is not populated
>>  * - Kerberos service support is not enabled
>>  * - openid connect is not enabled
>>  * - knox sso is not enabled
>>  * </p>
>>  *
>>  * @return true if client certificates are required for access to the REST API
>>  */
>> public boolean isClientAuthRequiredForRestApi() {
>>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
>> }
>> There is active planning for a complete rewrite of the authentication mechanism ordering, but because this is such a wide-reaching change and will have substantial impact across the project, I strongly advocate for this to go in a major release. As the number of users of the project continues to grow, we have to balance improving the experience in edge scenarios against more common scenarios. While I don’t mean to negate the times a reverse proxy is required, it is not present in the majority of deployments, and the current model works sufficiently in those deployments. We have to keep those deployments in mind as well while we make these changes.
>> 
>> If you have an immediate need to deploy NiFi behind a reverse proxy and disable the X.509 requests, my honest suggestion would be to fork the repository and apply a patch to return false from that method just as you specified. Unfortunately, that patch alone probably would not be accepted into the upstream NiFi project at this time. It can certainly be documented as a requirement for the rearchitected authentication mechanism moving forward.
>> 
>> Thanks for sharing your expectations and needs from the project, and hopefully we can meet them soon.
>> 
>> 
>> Andy LoPresto
>> alopresto@apache.org <ma...@apache.org>
>> alopresto.apache@gmail.com <ma...@gmail.com>
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>>> 
>>> FYSA,
>>> 
>>> This is where X509 is "always-on".
>>> 
>>> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>>> 
>>> if (props.isClientAuthRequiredForRestApi()) {
>>>    contextFactory.setNeedClientAuth(true);
>>> } else {
>>>    contextFactory.setWantClientAuth(true);
>>> }
>>> 
>>> I believe in the short term, modifying this section to use nifi.properties to allow us to provide a false to wantClientAuth, would address our concerns.
>>> 
>>> --
>>> Curtis Ruck
>>> 
>>> 
>>> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>>> To support Shawn's statement even further.  If my customer can't get NiFi to operate behind our reverse proxy, it won't be in our system.  I'm trying to find the easiest approach, and NiFi's OIDC should be perfect, if X509 wasn't "wanted" up front.
>>> 
>>> I'd argue that all of the AuthN/AuthZ code should be abstracted out significantly more than it currently is, with the ability to completely configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>>> The ability to manage users/groups in NiFi's UI should be a toggle.
>>> There should be an easy higher level API to use for group/role provisioning.  If a new user "bob" open's NiFi and they have a "read-only" role, then they shouldn't need to be manually provisioned in NiFi, and we my customer tries to minimize the number of unique applications reaching into LDAP.  Every application that implements LDAP support implements it differently, and they don't always scale up appropriately.
>>> 
>>> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.  With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's custom protocols, which can support Yubikey, Google Authentication, ADFS, Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any of it.
>>> 
>>> I'm even willing to assist in providing some PRs to move NiFi in the right direction, I just think we should figure out the higher level architecture/design a little better; especially since NiFi's job is to help things integrate together, it's not being a good team player.
>>> 
>>> As much as I hate to say it, if NiFi was a proper Java EE project, I could just use a war overlay to modify the AuthN/AuthZ to success; even if it was just a self-executing .war.
>>> 
>>> --
>>> Curtis Ruck
>>> 
>>> 
>>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
>>> I'll clarify my statement a little as well with a workflow.
>>> 
>>> You open the NiFi UI Link
>>> Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>> Then you get Prompts for Username and Password because of GSSAPI even though your not on that REALM.
>>> Then you get directed to the Identify Management Reverse Proxy URL for Knox SSO
>>> Then you get prompted for your Certificate which you should select.
>>> Then you might get prompted for Kerberos Again which you cancel
>>> Finally your in NiFi.
>>> 
>>> Painful doesn't even begin to describe it lol.
>>> 
>>> 
>>> Thanks
>>> Shawn
>>> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
>>> Sent: Thursday, August 9, 2018 11:07:28 AM
>>> To: users@nifi.apache.org <ma...@nifi.apache.org>
>>> Subject: Re: Re:
>>> 
>>> Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful
>>> 
>>> Wow, that sounds terrible. Confusing, accident prone, and frustrating to correct mistakes (at least in my experience, forcing a browser to forget client certificate preferences is difficult).
>>> 
>>> Thanks for sharing those details about your deployment scenario. This can definitely be improved and I have some ideas for how to do it. I've cloned the issue to NiFi to make sure we are tracking it for both projects [1][2]
>>> 
>>> [1] https://issues.apache.org/jira/browse/NIFIREG-189 <https://issues.apache.org/jira/browse/NIFIREG-189>
>>> [2] https://issues.apache.org/jira/browse/NIFI-5504 <https://issues.apache.org/jira/browse/NIFI-5504>
>>> 
>>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
>>> The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.
>>> 
>>> We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.
>>> 
>>> Thanks
>>> Shawnk
>>> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
>>> Sent: Thursday, August 9, 2018 10:47:56 AM
>>> To: users@nifi.apache.org <ma...@nifi.apache.org>
>>> Subject: Re:
>>> 
>>> sorry forgot the link. here it is:
>>> 
>>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189 <https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189>
>>> 
>>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kdoran@apache.org <ma...@apache.org>> wrote:
>>> Hi Curtis,
>>> 
>>> This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.
>>> 
>>> Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?
>>> 
>>> Thanks,
>>> Kevin
>>> 
>>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>>> I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?
>>> 
>>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>> 
>>> --
>>> Curtis Ruck
>>> 
>>> 
>>> 
>> 
> 

Re:

[Andy LoPresto <al...@apache.org>]

I think we agree in our assessment of what the code is doing and disagree in our desire for how that should occur. If OIDC is enabled and isClientAuthRequiredForRestApi() returns false, the result is:

// Functionally equivalent to contextFactory.setNeedClientAuth(false);
contextFactory.setWantClientAuth(true);

That means that the server will request a client certificate if available, but will not require its presence to negotiate the TLS handshake. You are asking to set contextFactory.setWantClientAuth(false); as well, which will suppress the certificate selection dialog. If needClientAuth and wantClientAuth are both false, client certificates cannot be used to authenticate as they will never be sent from the browser. This will effectively allow you to choose between “certificates only”, “certificates or other provider”, and “other provider only (no certificates)”.

I am saying that core NiFi *always* accepts client certificates as an authentication mechanism; there is no scenario in which need and want are both set to false. This is by design. Again, I am not saying this can never change, but because of the expectations, documentation, and shared knowledge around this mechanism, changing it is (in my opinion) a major change, and should not be done in a minor release. Other project members may (and probably do) disagree with me.

A property in nifi.properties which defaults to “off” but when manually enabled can bypass this requirement is an option. I don’t think we disagree on how to implement this specific change; I think we differ only on how impactful it will be. My perspective comes from supporting a large number of users with a broad variety of (often conflicting) requirements, and sometimes (both they and I have) very little knowledge of the rest of their IT ecosystem. I believe your perspective comes from a specific user with specific requirements. That’s why I recommended making the localized change you need in a fork of the project, so you can achieve your objective in a timeframe that is not blocked by other parties.

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 9, 2018, at 11:48 AM, Curtis Ruck <cu...@gmail.com> wrote:
> 
> In my environment, by trying to enable OIDC, it returns false in that function you selected.
> 
> My suggestion, is that in the } else { block, changing the setWantClientAuth(true) to setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to true in the absence of the setting.
> 
> By allowing a property to disable this check, would neuter the current X509 Authentication, as it won't have a certificate to authenticate.  It would also address Shawn's concern of for having his users cancel the first authentication popup.
> 
> It's not fixing AuthN/AuthZ, but its allowing us in special circumstances to disable X509 easily.  In my environment, it's even preferable because we would use OIDC to redirect to Apereo CAS, which does X509 Authentication itself.
> 
> --
> Curtis Ruck
> 
> 
> On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <alopresto@apache.org <ma...@apache.org>> wrote:
> Hi Curtis,
> 
> There has definitely been some discussion about this and it is picking up recently. I understand the difficulty faced when using NiFi in conjunction with a reverse proxy or external identity provider. I am not saying the current way is perfect, or even the best.
> 
> …but…
> 
> Always allowing X.509 authentication has been the standard for years (due to NiFi’s original design requirements), and changing this will have far-reaching impact on the application. The additional identity providers have been added piece-meal as requirements arose, and because of the multiple contexts NiFi supports, there are additional changes required for each — issuing one time tokens for file downloads, etc.
> 
> The code you identified returns a boolean value, but it is not simply reading a boolean flag from the properties file — it is calculating the presence/absence of the other (non-credential-based) identity providers:
> 
> /**
>  * Returns true if client certificates are required for REST API. Determined
>  * if the following conditions are all true:
>  * <p>
>  * - login identity provider is not populated
>  * - Kerberos service support is not enabled
>  * - openid connect is not enabled
>  * - knox sso is not enabled
>  * </p>
>  *
>  * @return true if client certificates are required for access to the REST API
>  */
> public boolean isClientAuthRequiredForRestApi() {
>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
> }
> There is active planning for a complete rewrite of the authentication mechanism ordering, but because this is such a wide-reaching change and will have substantial impact across the project, I strongly advocate for this to go in a major release. As the number of users of the project continues to grow, we have to balance improving the experience in edge scenarios against more common scenarios. While I don’t mean to negate the times a reverse proxy is required, it is not present in the majority of deployments, and the current model works sufficiently in those deployments. We have to keep those deployments in mind as well while we make these changes.
> 
> If you have an immediate need to deploy NiFi behind a reverse proxy and disable the X.509 requests, my honest suggestion would be to fork the repository and apply a patch to return false from that method just as you specified. Unfortunately, that patch alone probably would not be accepted into the upstream NiFi project at this time. It can certainly be documented as a requirement for the rearchitected authentication mechanism moving forward.
> 
> Thanks for sharing your expectations and needs from the project, and hopefully we can meet them soon.
> 
> 
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com <ma...@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>> 
>> FYSA,
>> 
>> This is where X509 is "always-on".
>> 
>> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>> 
>> if (props.isClientAuthRequiredForRestApi()) {
>>    contextFactory.setNeedClientAuth(true);
>> } else {
>>    contextFactory.setWantClientAuth(true);
>> }
>> 
>> I believe in the short term, modifying this section to use nifi.properties to allow us to provide a false to wantClientAuth, would address our concerns.
>> 
>> --
>> Curtis Ruck
>> 
>> 
>> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>> To support Shawn's statement even further.  If my customer can't get NiFi to operate behind our reverse proxy, it won't be in our system.  I'm trying to find the easiest approach, and NiFi's OIDC should be perfect, if X509 wasn't "wanted" up front.
>> 
>> I'd argue that all of the AuthN/AuthZ code should be abstracted out significantly more than it currently is, with the ability to completely configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>> The ability to manage users/groups in NiFi's UI should be a toggle.
>> There should be an easy higher level API to use for group/role provisioning.  If a new user "bob" open's NiFi and they have a "read-only" role, then they shouldn't need to be manually provisioned in NiFi, and we my customer tries to minimize the number of unique applications reaching into LDAP.  Every application that implements LDAP support implements it differently, and they don't always scale up appropriately.
>> 
>> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.  With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's custom protocols, which can support Yubikey, Google Authentication, ADFS, Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any of it.
>> 
>> I'm even willing to assist in providing some PRs to move NiFi in the right direction, I just think we should figure out the higher level architecture/design a little better; especially since NiFi's job is to help things integrate together, it's not being a good team player.
>> 
>> As much as I hate to say it, if NiFi was a proper Java EE project, I could just use a war overlay to modify the AuthN/AuthZ to success; even if it was just a self-executing .war.
>> 
>> --
>> Curtis Ruck
>> 
>> 
>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
>> I'll clarify my statement a little as well with a workflow.
>> 
>> You open the NiFi UI Link
>> Chrome sees NiFi Asking for SSL and Prompts You for Cert
>> Then you get Prompts for Username and Password because of GSSAPI even though your not on that REALM.
>> Then you get directed to the Identify Management Reverse Proxy URL for Knox SSO
>> Then you get prompted for your Certificate which you should select.
>> Then you might get prompted for Kerberos Again which you cancel
>> Finally your in NiFi.
>> 
>> Painful doesn't even begin to describe it lol.
>> 
>> 
>> Thanks
>> Shawn
>> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
>> Sent: Thursday, August 9, 2018 11:07:28 AM
>> To: users@nifi.apache.org <ma...@nifi.apache.org>
>> Subject: Re: Re:
>> 
>> Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful
>> 
>> Wow, that sounds terrible. Confusing, accident prone, and frustrating to correct mistakes (at least in my experience, forcing a browser to forget client certificate preferences is difficult).
>> 
>> Thanks for sharing those details about your deployment scenario. This can definitely be improved and I have some ideas for how to do it. I've cloned the issue to NiFi to make sure we are tracking it for both projects [1][2]
>> 
>> [1] https://issues.apache.org/jira/browse/NIFIREG-189 <https://issues.apache.org/jira/browse/NIFIREG-189>
>> [2] https://issues.apache.org/jira/browse/NIFI-5504 <https://issues.apache.org/jira/browse/NIFI-5504>
>> 
>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
>> The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.
>> 
>> We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.
>> 
>> Thanks
>> Shawnk
>> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
>> Sent: Thursday, August 9, 2018 10:47:56 AM
>> To: users@nifi.apache.org <ma...@nifi.apache.org>
>> Subject: Re:
>> 
>> sorry forgot the link. here it is:
>> 
>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189 <https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189>
>> 
>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kdoran@apache.org <ma...@apache.org>> wrote:
>> Hi Curtis,
>> 
>> This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.
>> 
>> Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?
>> 
>> Thanks,
>> Kevin
>> 
>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
>> I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?
>> 
>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>> 
>> --
>> Curtis Ruck
>> 
>> 
>> 
> 

Re:

[Curtis Ruck <cu...@gmail.com>]

In my environment, by trying to enable OIDC, it returns false in that
function you selected.

My suggestion, is that in the } else { block, changing the
setWantClientAuth(true) to
setWantClientAuth(nifiproperties.isWantClientAuth()) which can default to
true in the absence of the setting.

By allowing a property to disable this check, would neuter the current X509
Authentication, as it won't have a certificate to authenticate.  It would
also address Shawn's concern of for having his users cancel the first
authentication popup.

It's not fixing AuthN/AuthZ, but its allowing us in special circumstances
to disable X509 easily.  In my environment, it's even preferable because we
would use OIDC to redirect to Apereo CAS, which does X509 Authentication
itself.

--
Curtis Ruck


On Thu, Aug 9, 2018 at 2:43 PM Andy LoPresto <al...@apache.org> wrote:

> Hi Curtis,
>
> There has definitely been some discussion about this and it is picking up
> recently. I understand the difficulty faced when using NiFi in conjunction
> with a reverse proxy or external identity provider. I am not saying the
> current way is perfect, or even the best.
>
> …but…
>
> Always allowing X.509 authentication has been the standard for years (due
> to NiFi’s original design requirements), and changing this will have
> far-reaching impact on the application. The additional identity providers
> have been added piece-meal as requirements arose, and because of the
> multiple contexts NiFi supports, there are additional changes required for
> each — issuing one time tokens for file downloads, etc.
>
> The code you identified returns a boolean value, but it is not simply
> reading a boolean flag from the properties file — it is calculating the
> presence/absence of the other (non-credential-based) identity providers:
>
> /**
>  * Returns true if client certificates are required for REST API. Determined
>  * if the following conditions are all true:
>  * <p>
>  * - login identity provider is not populated
>  * - Kerberos service support is not enabled
>  * - openid connect is not enabled
>  * - knox sso is not enabled
>  * </p>
>  *
>  * @return true if client certificates are required for access to the REST API
>  */
> public boolean isClientAuthRequiredForRestApi() {
>     return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
> }
>
> There is active planning for a complete rewrite of the authentication
> mechanism ordering, but because this is such a wide-reaching change and
> will have substantial impact across the project, I strongly advocate for
> this to go in a major release. As the number of users of the project
> continues to grow, we have to balance improving the experience in edge
> scenarios against more common scenarios. While I don’t mean to negate the
> times a reverse proxy is required, it is not present in the majority of
> deployments, and the current model works sufficiently in those deployments.
> We have to keep those deployments in mind as well while we make these
> changes.
>
> If you have an immediate need to deploy NiFi behind a reverse proxy and
> disable the X.509 requests, my honest suggestion would be to fork the
> repository and apply a patch to return false from that method just as you
> specified. Unfortunately, that patch alone probably would not be accepted
> into the upstream NiFi project at this time. It can certainly be documented
> as a requirement for the rearchitected authentication mechanism moving
> forward.
>
> Thanks for sharing your expectations and needs from the project, and
> hopefully we can meet them soon.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <cu...@gmail.com> wrote:
>
> FYSA,
>
> This is where X509 is "always-on".
>
>
> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
>
> if (props.isClientAuthRequiredForRestApi()) {
>    contextFactory.setNeedClientAuth(true);
> } else {
>    contextFactory.setWantClientAuth(true);
> }
>
> I believe in the short term, modifying this section to use nifi.properties
> to allow us to provide a false to wantClientAuth, would address our
> concerns.
>
> --
> Curtis Ruck
>
>
> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <cu...@gmail.com> wrote:
>
>> To support Shawn's statement even further.  If my customer can't get NiFi
>> to operate behind our reverse proxy, it won't be in our system.  I'm trying
>> to find the easiest approach, and NiFi's OIDC should be perfect, if X509
>> wasn't "wanted" up front.
>>
>> I'd argue that all of the AuthN/AuthZ code should be abstracted out
>> significantly more than it currently is, with the ability to completely
>> configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
>> The ability to manage users/groups in NiFi's UI should be a toggle.
>> There should be an easy higher level API to use for group/role
>> provisioning.  If a new user "bob" open's NiFi and they have a "read-only"
>> role, then they shouldn't need to be manually provisioned in NiFi, and we
>> my customer tries to minimize the number of unique applications reaching
>> into LDAP.  Every application that implements LDAP support implements it
>> differently, and they don't always scale up appropriately.
>>
>> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.
>> With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's
>> custom protocols, which can support Yubikey, Google Authentication, ADFS,
>> Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any
>> of it.
>>
>> I'm even willing to assist in providing some PRs to move NiFi in the
>> right direction, I just think we should figure out the higher level
>> architecture/design a little better; especially since NiFi's job is to help
>> things integrate together, it's not being a good team player.
>>
>> As much as I hate to say it, if NiFi was a proper Java EE project, I
>> could just use a war overlay to modify the AuthN/AuthZ to success; even if
>> it was just a self-executing .war.
>>
>> --
>> Curtis Ruck
>>
>>
>> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us>
>> wrote:
>>
>>> I'll clarify my statement a little as well with a workflow.
>>>
>>>
>>>    1. You open the NiFi UI Link
>>>    2. Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>>    3. Then you get Prompts for Username and Password because of GSSAPI
>>>    even though your not on that REALM.
>>>    4. Then you get directed to the Identify Management Reverse Proxy
>>>    URL for Knox SSO
>>>    5. Then you get prompted for your Certificate which you should
>>>    select.
>>>    6. Then you might get prompted for Kerberos Again which you cancel
>>>    7. Finally your in NiFi.
>>>
>>>
>>> Painful doesn't even begin to describe it lol.
>>>
>>>
>>> Thanks
>>> Shawn
>>> ------------------------------
>>> *From:* Kevin Doran <kd...@apache.org>
>>> *Sent:* Thursday, August 9, 2018 11:07:28 AM
>>> *To:* users@nifi.apache.org
>>> *Subject:* Re: Re:
>>>
>>>
>>> *Explaining to your end users that you should skip the first Certificate
>>> Prompt but accept the second but only when you haven't logged in the
>>> current session is really painful*
>>>
>>>
>>> Wow, that sounds terrible. Confusing, accident prone, and frustrating to
>>> correct mistakes (at least in my experience, forcing a browser to forget
>>> client certificate preferences is difficult).
>>>
>>> Thanks for sharing those details about your deployment scenario. This
>>> can definitely be improved and I have some ideas for how to do it. I've
>>> cloned the issue to NiFi to make sure we are tracking it for both projects
>>> [1][2]
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFIREG-189
>>> [2] https://issues.apache.org/jira/browse/NIFI-5504
>>>
>>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us>
>>> wrote:
>>>
>>> The project I'm on is running into this issue as well and it gets
>>> particularly painful when all of your server's are signed by the same root
>>> ca that signs your smart card logins and your using something like KnoxSSO.
>>> Explaining to your end users that you should skip the first Certificate
>>> Prompt but accept the second but only when you haven't logged in the
>>> current session is really painful and shows major shortcoming between the
>>> back end authentication between servers and front end ui authentication.
>>>
>>> We can't even considering putting it behind our identify reverse proxies
>>> because we can't turn off two way ssl.
>>>
>>> Thanks
>>> Shawnk
>>> ------------------------------
>>> *From:* Kevin Doran <kd...@apache.org>
>>> *Sent:* Thursday, August 9, 2018 10:47:56 AM
>>> *To:* users@nifi.apache.org
>>> *Subject:* Re:
>>>
>>> sorry forgot the link. here it is:
>>>
>>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>>>
>>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>>>
>>> Hi Curtis,
>>>
>>> This has come up a few times. Unfortunately I don’t think there is
>>> currently an easy way to disable X509-based identity extraction in NiFi
>>> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
>>> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
>>> NiFi, so this ticket should apply to NiFi as well.
>>>
>>> Perhaps you could share more about your needs and use case on that
>>> ticket so that when it gets implemented we could take that scenario with
>>> reverse proxies and OIDC into account?
>>>
>>> Thanks,
>>> Kevin
>>>
>>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
>>> wrote:
>>>
>>> I'm trying to setup OIDC authentication, but with Nifi service existing
>>> behind a reverse proxy, and for our other apps we use SSL Client
>>> Authentication between reverse proxy and application, Nifi is picking up
>>> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
>>> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>>>
>>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>>
>>> --
>>> Curtis Ruck
>>>
>>>
>>>
>>>
>>>
>

Re:

[Andy LoPresto <al...@apache.org>]

Hi Curtis,

There has definitely been some discussion about this and it is picking up recently. I understand the difficulty faced when using NiFi in conjunction with a reverse proxy or external identity provider. I am not saying the current way is perfect, or even the best.

…but…

Always allowing X.509 authentication has been the standard for years (due to NiFi’s original design requirements), and changing this will have far-reaching impact on the application. The additional identity providers have been added piece-meal as requirements arose, and because of the multiple contexts NiFi supports, there are additional changes required for each — issuing one time tokens for file downloads, etc.

The code you identified returns a boolean value, but it is not simply reading a boolean flag from the properties file — it is calculating the presence/absence of the other (non-credential-based) identity providers:

/**
 * Returns true if client certificates are required for REST API. Determined
 * if the following conditions are all true:
 * <p>
 * - login identity provider is not populated
 * - Kerberos service support is not enabled
 * - openid connect is not enabled
 * - knox sso is not enabled
 * </p>
 *
 * @return true if client certificates are required for access to the REST API
 */
public boolean isClientAuthRequiredForRestApi() {
    return !isLoginIdentityProviderEnabled() && !isKerberosSpnegoSupportEnabled() && !isOidcEnabled() && !isKnoxSsoEnabled();
}
There is active planning for a complete rewrite of the authentication mechanism ordering, but because this is such a wide-reaching change and will have substantial impact across the project, I strongly advocate for this to go in a major release. As the number of users of the project continues to grow, we have to balance improving the experience in edge scenarios against more common scenarios. While I don’t mean to negate the times a reverse proxy is required, it is not present in the majority of deployments, and the current model works sufficiently in those deployments. We have to keep those deployments in mind as well while we make these changes.

If you have an immediate need to deploy NiFi behind a reverse proxy and disable the X.509 requests, my honest suggestion would be to fork the repository and apply a patch to return false from that method just as you specified. Unfortunately, that patch alone probably would not be accepted into the upstream NiFi project at this time. It can certainly be documented as a requirement for the rearchitected authentication mechanism moving forward.

Thanks for sharing your expectations and needs from the project, and hopefully we can meet them soon.


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 9, 2018, at 11:28 AM, Curtis Ruck <cu...@gmail.com> wrote:
> 
> FYSA,
> 
> This is where X509 is "always-on".
> 
> nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785
> 
> if (props.isClientAuthRequiredForRestApi()) {
>    contextFactory.setNeedClientAuth(true);
> } else {
>    contextFactory.setWantClientAuth(true);
> }
> 
> I believe in the short term, modifying this section to use nifi.properties to allow us to provide a false to wantClientAuth, would address our concerns.
> 
> --
> Curtis Ruck
> 
> 
> On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
> To support Shawn's statement even further.  If my customer can't get NiFi to operate behind our reverse proxy, it won't be in our system.  I'm trying to find the easiest approach, and NiFi's OIDC should be perfect, if X509 wasn't "wanted" up front.
> 
> I'd argue that all of the AuthN/AuthZ code should be abstracted out significantly more than it currently is, with the ability to completely configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
> The ability to manage users/groups in NiFi's UI should be a toggle.
> There should be an easy higher level API to use for group/role provisioning.  If a new user "bob" open's NiFi and they have a "read-only" role, then they shouldn't need to be manually provisioned in NiFi, and we my customer tries to minimize the number of unique applications reaching into LDAP.  Every application that implements LDAP support implements it differently, and they don't always scale up appropriately.
> 
> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.  With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's custom protocols, which can support Yubikey, Google Authentication, ADFS, Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any of it.
> 
> I'm even willing to assist in providing some PRs to move NiFi in the right direction, I just think we should figure out the higher level architecture/design a little better; especially since NiFi's job is to help things integrate together, it's not being a good team player.
> 
> As much as I hate to say it, if NiFi was a proper Java EE project, I could just use a war overlay to modify the AuthN/AuthZ to success; even if it was just a self-executing .war.
> 
> --
> Curtis Ruck
> 
> 
> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
> I'll clarify my statement a little as well with a workflow.
> 
> You open the NiFi UI Link
> Chrome sees NiFi Asking for SSL and Prompts You for Cert
> Then you get Prompts for Username and Password because of GSSAPI even though your not on that REALM.
> Then you get directed to the Identify Management Reverse Proxy URL for Knox SSO
> Then you get prompted for your Certificate which you should select.
> Then you might get prompted for Kerberos Again which you cancel
> Finally your in NiFi.
> 
> Painful doesn't even begin to describe it lol.
> 
> 
> Thanks
> Shawn
> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
> Sent: Thursday, August 9, 2018 11:07:28 AM
> To: users@nifi.apache.org <ma...@nifi.apache.org>
> Subject: Re: Re:
> 
> Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful
> 
> Wow, that sounds terrible. Confusing, accident prone, and frustrating to correct mistakes (at least in my experience, forcing a browser to forget client certificate preferences is difficult).
> 
> Thanks for sharing those details about your deployment scenario. This can definitely be improved and I have some ideas for how to do it. I've cloned the issue to NiFi to make sure we are tracking it for both projects [1][2]
> 
> [1] https://issues.apache.org/jira/browse/NIFIREG-189 <https://issues.apache.org/jira/browse/NIFIREG-189>
> [2] https://issues.apache.org/jira/browse/NIFI-5504 <https://issues.apache.org/jira/browse/NIFI-5504>
> 
> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sweeks@weeksconsulting.us <ma...@weeksconsulting.us>> wrote:
> The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.
> 
> We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.
> 
> Thanks
> Shawnk
> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
> Sent: Thursday, August 9, 2018 10:47:56 AM
> To: users@nifi.apache.org <ma...@nifi.apache.org>
> Subject: Re:
> 
> sorry forgot the link. here it is:
> 
> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189 <https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189>
> 
> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kdoran@apache.org <ma...@apache.org>> wrote:
> Hi Curtis,
> 
> This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.
> 
> Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?
> 
> Thanks,
> Kevin
> 
> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <curtis.ruck@gmail.com <ma...@gmail.com>> wrote:
> I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?
> 
> Connecting directly to nifi, it triggers the proper OIDC redirects.
> 
> --
> Curtis Ruck
> 
> 
> 

Re: Re:

[Curtis Ruck <cu...@gmail.com>]

FYSA,

This is where X509 is "always-on".

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L781-L785

if (props.isClientAuthRequiredForRestApi()) {
   contextFactory.setNeedClientAuth(true);
} else {
   contextFactory.setWantClientAuth(true);
}

I believe in the short term, modifying this section to use nifi.properties
to allow us to provide a false to wantClientAuth, would address our
concerns.

--
Curtis Ruck


On Thu, Aug 9, 2018 at 12:54 PM Curtis Ruck <cu...@gmail.com> wrote:

> To support Shawn's statement even further.  If my customer can't get NiFi
> to operate behind our reverse proxy, it won't be in our system.  I'm trying
> to find the easiest approach, and NiFi's OIDC should be perfect, if X509
> wasn't "wanted" up front.
>
> I'd argue that all of the AuthN/AuthZ code should be abstracted out
> significantly more than it currently is, with the ability to completely
> configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
> The ability to manage users/groups in NiFi's UI should be a toggle.
> There should be an easy higher level API to use for group/role
> provisioning.  If a new user "bob" open's NiFi and they have a "read-only"
> role, then they shouldn't need to be manually provisioned in NiFi, and we
> my customer tries to minimize the number of unique applications reaching
> into LDAP.  Every application that implements LDAP support implements it
> differently, and they don't always scale up appropriately.
>
> For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.
> With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's
> custom protocols, which can support Yubikey, Google Authentication, ADFS,
> Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any
> of it.
>
> I'm even willing to assist in providing some PRs to move NiFi in the right
> direction, I just think we should figure out the higher level
> architecture/design a little better; especially since NiFi's job is to help
> things integrate together, it's not being a good team player.
>
> As much as I hate to say it, if NiFi was a proper Java EE project, I could
> just use a war overlay to modify the AuthN/AuthZ to success; even if it was
> just a self-executing .war.
>
> --
> Curtis Ruck
>
>
> On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us>
> wrote:
>
>> I'll clarify my statement a little as well with a workflow.
>>
>>
>>
>>    1. You open the NiFi UI Link
>>    2. Chrome sees NiFi Asking for SSL and Prompts You for Cert
>>    3. Then you get Prompts for Username and Password because of GSSAPI
>>    even though your not on that REALM.
>>    4. Then you get directed to the Identify Management Reverse Proxy URL
>>    for Knox SSO
>>    5. Then you get prompted for your Certificate which you should select.
>>    6. Then you might get prompted for Kerberos Again which you cancel
>>    7. Finally your in NiFi.
>>
>>
>> Painful doesn't even begin to describe it lol.
>>
>>
>> Thanks
>>
>> Shawn
>> ------------------------------
>> *From:* Kevin Doran <kd...@apache.org>
>> *Sent:* Thursday, August 9, 2018 11:07:28 AM
>> *To:* users@nifi.apache.org
>> *Subject:* Re: Re:
>>
>>
>> *Explaining to your end users that you should skip the first Certificate
>> Prompt but accept the second but only when you haven't logged in the
>> current session is really painful*
>>
>>
>> Wow, that sounds terrible. Confusing, accident prone, and frustrating to
>> correct mistakes (at least in my experience, forcing a browser to forget
>> client certificate preferences is difficult).
>>
>> Thanks for sharing those details about your deployment scenario. This can
>> definitely be improved and I have some ideas for how to do it. I've cloned
>> the issue to NiFi to make sure we are tracking it for both projects [1][2]
>>
>> [1] https://issues.apache.org/jira/browse/NIFIREG-189
>> [2] https://issues.apache.org/jira/browse/NIFI-5504
>>
>> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us>
>> wrote:
>>
>> The project I'm on is running into this issue as well and it gets
>> particularly painful when all of your server's are signed by the same root
>> ca that signs your smart card logins and your using something like KnoxSSO.
>> Explaining to your end users that you should skip the first Certificate
>> Prompt but accept the second but only when you haven't logged in the
>> current session is really painful and shows major shortcoming between the
>> back end authentication between servers and front end ui authentication.
>>
>>
>> We can't even considering putting it behind our identify reverse proxies
>> because we can't turn off two way ssl.
>>
>>
>> Thanks
>>
>> Shawnk
>> ------------------------------
>> *From:* Kevin Doran <kd...@apache.org>
>> *Sent:* Thursday, August 9, 2018 10:47:56 AM
>> *To:* users@nifi.apache.org
>> *Subject:* Re:
>>
>> sorry forgot the link. here it is:
>>
>> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>>
>> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>>
>> Hi Curtis,
>>
>> This has come up a few times. Unfortunately I don’t think there is
>> currently an easy way to disable X509-based identity extraction in NiFi
>> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
>> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
>> NiFi, so this ticket should apply to NiFi as well.
>>
>> Perhaps you could share more about your needs and use case on that ticket
>> so that when it gets implemented we could take that scenario with reverse
>> proxies and OIDC into account?
>>
>> Thanks,
>> Kevin
>>
>> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
>> wrote:
>>
>> I'm trying to setup OIDC authentication, but with Nifi service existing
>> behind a reverse proxy, and for our other apps we use SSL Client
>> Authentication between reverse proxy and application, Nifi is picking up
>> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
>> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>>
>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>
>> --
>> Curtis Ruck
>>
>>
>>
>>
>>

Re: Re:

[Curtis Ruck <cu...@gmail.com>]

To support Shawn's statement even further.  If my customer can't get NiFi
to operate behind our reverse proxy, it won't be in our system.  I'm trying
to find the easiest approach, and NiFi's OIDC should be perfect, if X509
wasn't "wanted" up front.

I'd argue that all of the AuthN/AuthZ code should be abstracted out
significantly more than it currently is, with the ability to completely
configure it via nifi.properties, and mix-in custom AuthN/AuthZ solutions.
The ability to manage users/groups in NiFi's UI should be a toggle.
There should be an easy higher level API to use for group/role
provisioning.  If a new user "bob" open's NiFi and they have a "read-only"
role, then they shouldn't need to be manually provisioned in NiFi, and we
my customer tries to minimize the number of unique applications reaching
into LDAP.  Every application that implements LDAP support implements it
differently, and they don't always scale up appropriately.

For example, i'm trying to get Apereo CAS 5.x working with Apache NiFi.
With CAS, it can provide SAML 2.0, SAML 1.1, OpenID Connect, or CAS's
custom protocols, which can support Yubikey, Google Authentication, ADFS,
Azure AD, etc.  Sadly, because of the wantClientAuth(true) I can't use any
of it.

I'm even willing to assist in providing some PRs to move NiFi in the right
direction, I just think we should figure out the higher level
architecture/design a little better; especially since NiFi's job is to help
things integrate together, it's not being a good team player.

As much as I hate to say it, if NiFi was a proper Java EE project, I could
just use a war overlay to modify the AuthN/AuthZ to success; even if it was
just a self-executing .war.

--
Curtis Ruck


On Thu, Aug 9, 2018 at 12:14 PM Shawn Weeks <sw...@weeksconsulting.us>
wrote:

> I'll clarify my statement a little as well with a workflow.
>
>
>
>    1. You open the NiFi UI Link
>    2. Chrome sees NiFi Asking for SSL and Prompts You for Cert
>    3. Then you get Prompts for Username and Password because of GSSAPI
>    even though your not on that REALM.
>    4. Then you get directed to the Identify Management Reverse Proxy URL
>    for Knox SSO
>    5. Then you get prompted for your Certificate which you should select.
>    6. Then you might get prompted for Kerberos Again which you cancel
>    7. Finally your in NiFi.
>
>
> Painful doesn't even begin to describe it lol.
>
>
> Thanks
>
> Shawn
> ------------------------------
> *From:* Kevin Doran <kd...@apache.org>
> *Sent:* Thursday, August 9, 2018 11:07:28 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: Re:
>
>
> *Explaining to your end users that you should skip the first Certificate
> Prompt but accept the second but only when you haven't logged in the
> current session is really painful*
>
>
> Wow, that sounds terrible. Confusing, accident prone, and frustrating to
> correct mistakes (at least in my experience, forcing a browser to forget
> client certificate preferences is difficult).
>
> Thanks for sharing those details about your deployment scenario. This can
> definitely be improved and I have some ideas for how to do it. I've cloned
> the issue to NiFi to make sure we are tracking it for both projects [1][2]
>
> [1] https://issues.apache.org/jira/browse/NIFIREG-189
> [2] https://issues.apache.org/jira/browse/NIFI-5504
>
> On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us>
> wrote:
>
> The project I'm on is running into this issue as well and it gets
> particularly painful when all of your server's are signed by the same root
> ca that signs your smart card logins and your using something like KnoxSSO.
> Explaining to your end users that you should skip the first Certificate
> Prompt but accept the second but only when you haven't logged in the
> current session is really painful and shows major shortcoming between the
> back end authentication between servers and front end ui authentication.
>
>
> We can't even considering putting it behind our identify reverse proxies
> because we can't turn off two way ssl.
>
>
> Thanks
>
> Shawnk
> ------------------------------
> *From:* Kevin Doran <kd...@apache.org>
> *Sent:* Thursday, August 9, 2018 10:47:56 AM
> *To:* users@nifi.apache.org
> *Subject:* Re:
>
> sorry forgot the link. here it is:
>
> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>
> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>
> Hi Curtis,
>
> This has come up a few times. Unfortunately I don’t think there is
> currently an easy way to disable X509-based identity extraction in NiFi
> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
> NiFi, so this ticket should apply to NiFi as well.
>
> Perhaps you could share more about your needs and use case on that ticket
> so that when it gets implemented we could take that scenario with reverse
> proxies and OIDC into account?
>
> Thanks,
> Kevin
>
> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
> wrote:
>
> I'm trying to setup OIDC authentication, but with Nifi service existing
> behind a reverse proxy, and for our other apps we use SSL Client
> Authentication between reverse proxy and application, Nifi is picking up
> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>
> Connecting directly to nifi, it triggers the proper OIDC redirects.
>
> --
> Curtis Ruck
>
>
>
>
>

Re: Re:

[Shawn Weeks <sw...@weeksconsulting.us>]

I'll clarify my statement a little as well with a workflow.


  1.  You open the NiFi UI Link
  2.  Chrome sees NiFi Asking for SSL and Prompts You for Cert
  3.  Then you get Prompts for Username and Password because of GSSAPI even though your not on that REALM.
  4.  Then you get directed to the Identify Management Reverse Proxy URL for Knox SSO
  5.  Then you get prompted for your Certificate which you should select.
  6.  Then you might get prompted for Kerberos Again which you cancel
  7.  Finally your in NiFi.


Painful doesn't even begin to describe it lol.


Thanks

Shawn

________________________________
From: Kevin Doran <kd...@apache.org>
Sent: Thursday, August 9, 2018 11:07:28 AM
To: users@nifi.apache.org
Subject: Re: Re:

Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful

Wow, that sounds terrible. Confusing, accident prone, and frustrating to correct mistakes (at least in my experience, forcing a browser to forget client certificate preferences is difficult).

Thanks for sharing those details about your deployment scenario. This can definitely be improved and I have some ideas for how to do it. I've cloned the issue to NiFi to make sure we are tracking it for both projects [1][2]

[1] https://issues.apache.org/jira/browse/NIFIREG-189
[2] https://issues.apache.org/jira/browse/NIFI-5504

On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us>> wrote:

The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.


We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.


Thanks

Shawnk

________________________________
From: Kevin Doran <kd...@apache.org>>
Sent: Thursday, August 9, 2018 10:47:56 AM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Re:

sorry forgot the link. here it is:

[1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189

On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org>> wrote:
Hi Curtis,

This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.

Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?

Thanks,
Kevin

On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>> wrote:
I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?

Connecting directly to nifi, it triggers the proper OIDC redirects.

--
Curtis Ruck

Re: Re:

[Kevin Doran <kd...@apache.org>]

>
> *Explaining to your end users that you should skip the first Certificate
> Prompt but accept the second but only when you haven't logged in the
> current session is really painful*


Wow, that sounds terrible. Confusing, accident prone, and frustrating to
correct mistakes (at least in my experience, forcing a browser to forget
client certificate preferences is difficult).

Thanks for sharing those details about your deployment scenario. This can
definitely be improved and I have some ideas for how to do it. I've cloned
the issue to NiFi to make sure we are tracking it for both projects [1][2]

[1] https://issues.apache.org/jira/browse/NIFIREG-189
[2] https://issues.apache.org/jira/browse/NIFI-5504

On Thu, Aug 9, 2018 at 11:54 AM, Shawn Weeks <sw...@weeksconsulting.us>
wrote:

> The project I'm on is running into this issue as well and it gets
> particularly painful when all of your server's are signed by the same root
> ca that signs your smart card logins and your using something like KnoxSSO.
> Explaining to your end users that you should skip the first Certificate
> Prompt but accept the second but only when you haven't logged in the
> current session is really painful and shows major shortcoming between the
> back end authentication between servers and front end ui authentication.
>
>
> We can't even considering putting it behind our identify reverse proxies
> because we can't turn off two way ssl.
>
>
> Thanks
>
> Shawnk
> ------------------------------
> *From:* Kevin Doran <kd...@apache.org>
> *Sent:* Thursday, August 9, 2018 10:47:56 AM
> *To:* users@nifi.apache.org
> *Subject:* Re:
>
> sorry forgot the link. here it is:
>
> [1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189
>
> On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:
>
> Hi Curtis,
>
> This has come up a few times. Unfortunately I don’t think there is
> currently an easy way to disable X509-based identity extraction in NiFi
> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
> NiFi, so this ticket should apply to NiFi as well.
>
> Perhaps you could share more about your needs and use case on that ticket
> so that when it gets implemented we could take that scenario with reverse
> proxies and OIDC into account?
>
> Thanks,
> Kevin
>
> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
> wrote:
>
> I'm trying to setup OIDC authentication, but with Nifi service existing
> behind a reverse proxy, and for our other apps we use SSL Client
> Authentication between reverse proxy and application, Nifi is picking up
> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>
> Connecting directly to nifi, it triggers the proper OIDC redirects.
>
> --
> Curtis Ruck
>
>
>
>

Re: Re:

[Shawn Weeks <sw...@weeksconsulting.us>]

The project I'm on is running into this issue as well and it gets particularly painful when all of your server's are signed by the same root ca that signs your smart card logins and your using something like KnoxSSO. Explaining to your end users that you should skip the first Certificate Prompt but accept the second but only when you haven't logged in the current session is really painful and shows major shortcoming between the back end authentication between servers and front end ui authentication.


We can't even considering putting it behind our identify reverse proxies because we can't turn off two way ssl.


Thanks

Shawnk

________________________________
From: Kevin Doran <kd...@apache.org>
Sent: Thursday, August 9, 2018 10:47:56 AM
To: users@nifi.apache.org
Subject: Re:

sorry forgot the link. here it is:

[1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189

On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org>> wrote:
Hi Curtis,

This has come up a few times. Unfortunately I don’t think there is currently an easy way to disable X509-based identity extraction in NiFi today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi Registry follows the same AuthN/AuthZ design (and a fair amount of code) as NiFi, so this ticket should apply to NiFi as well.

Perhaps you could share more about your needs and use case on that ticket so that when it gets implemented we could take that scenario with reverse proxies and OIDC into account?

Thanks,
Kevin

On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>> wrote:
I'm trying to setup OIDC authentication, but with Nifi service existing behind a reverse proxy, and for our other apps we use SSL Client Authentication between reverse proxy and application, Nifi is picking up the Reverse Proxy's SSL Certificate and falling into X509 Authentication instead of OIDC.  Any idea how I can disable X509 authentication in Nifi?

Connecting directly to nifi, it triggers the proper OIDC redirects.

--
Curtis Ruck

Re:

[Kevin Doran <kd...@apache.org>]

sorry forgot the link. here it is:

[1] https://issues.apache.org/jira/projects/NIFIREG/issues/NIFIREG-189

On Thu, Aug 9, 2018 at 11:47 AM, Kevin Doran <kd...@apache.org> wrote:

> Hi Curtis,
>
> This has come up a few times. Unfortunately I don’t think there is
> currently an easy way to disable X509-based identity extraction in NiFi
> today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
> Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
> NiFi, so this ticket should apply to NiFi as well.
>
> Perhaps you could share more about your needs and use case on that ticket
> so that when it gets implemented we could take that scenario with reverse
> proxies and OIDC into account?
>
> Thanks,
> Kevin
>
> On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com>
> wrote:
>
>> I'm trying to setup OIDC authentication, but with Nifi service existing
>> behind a reverse proxy, and for our other apps we use SSL Client
>> Authentication between reverse proxy and application, Nifi is picking up
>> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
>> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>>
>> Connecting directly to nifi, it triggers the proper OIDC redirects.
>>
>> --
>> Curtis Ruck
>>
>
>

Re:

[Kevin Doran <kd...@apache.org>]

Hi Curtis,

This has come up a few times. Unfortunately I don’t think there is
currently an easy way to disable X509-based identity extraction in NiFi
today. There is an open JIRA for the same issue in NiFi Registry [1]. NiFi
Registry follows the same AuthN/AuthZ design (and a fair amount of code) as
NiFi, so this ticket should apply to NiFi as well.

Perhaps you could share more about your needs and use case on that ticket
so that when it gets implemented we could take that scenario with reverse
proxies and OIDC into account?

Thanks,
Kevin

On Mon, Aug 6, 2018 at 10:23 AM, Curtis Ruck <cu...@gmail.com> wrote:

> I'm trying to setup OIDC authentication, but with Nifi service existing
> behind a reverse proxy, and for our other apps we use SSL Client
> Authentication between reverse proxy and application, Nifi is picking up
> the Reverse Proxy's SSL Certificate and falling into X509 Authentication
> instead of OIDC. Any idea how I can disable X509 authentication in Nifi?
>
> Connecting directly to nifi, it triggers the proper OIDC redirects.
>
> --
> Curtis Ruck
>