You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Kevin Doran <kd...@apache.org> on 2018/03/16 13:58:17 UTC
Re: Error when sending HTTPS request to Nifi using the Restful API
When using access tokens, make sure your NiFi instance is configured to not require client certificates for the TLS certificates:
nifi.security.needClientAuth=false
This is because you only want the NiFi server (not the client) to establish its identity with a TLS certificate; the client will establish identity after the TLS connection is established by passing the username and password credentials that will be validated by the Login Identity Provider you configured for NiFi. For example:
nifi.security.user.login.identity.provider=ldap-provider
where “ldap-provider” is configured in login-identity-providers.xml
Hope this helps!
Kevin
From: Tian TD Deng <de...@cn.ibm.com>
Reply-To: <us...@nifi.apache.org>
Date: Friday, March 16, 2018 at 04:51
To: <us...@nifi.apache.org>
Subject: Error when sending HTTPS request to Nifi using the Restful API
Dear All,
I was trying to send HTTPS request to Nifi using the Restful API via curl, however, it didn't work as expected.
curl -k --tlsv1.2 https://localhost:8443/nifi-api/access/token --data 'username=test&password=password'
Could you please help to give some advice about this?
Thanks a lot!
Thanks & Best Regards
Joey(Tian) Deng
-------------------------------------------------------------------------------------------
IT Specialist
IBM China Global Delivery Center
Tel: (+86) 27 59269644
E-mail: dengtian@cn.ibm.com
WhatsApp: +86 15671158671
-------------------------------------------------------------------------------------------
Re: Error when sending HTTPS request to Nifi using the Restful API
Posted by Andy LoPresto <al...@apache.org>.
Also important to note that if no credential access is configured (LDAP, Kerberos, OIDC, Knox, etc.) and client certificate authentication is the only available mechanism, the access token endpoint is useless and will not generate a token, because every subsequent request will still require the client certificate to be present.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Mar 16, 2018, at 8:43 AM, Kevin Doran <kd...@apache.org> wrote:
>
> Any secured (HTTPS) NiFi instance requires client authentication for every REST API request, so the first thing you need to know is how your NiFi server is configured to authenticate clients. There are lots of options, including client certificates, login with LDAP username and password, or authenticate with Kerberos ticket, Knox identity, or OpenId connect. These options are explained in the NiFi Administration Guide [1]. One way to determine this is by looking at the files in the /conf directory (i.e., nifi.properties).
>
> Once you know how the NiFi server authenticates client requests, you must configure the REST API client you are using to perform the same authentication steps. This varies based on authentication mechanism, but is basically either a client certificate provided during the TLS connection handshake or obtaining a JWT access token using one of the /access/* REST API endpoints that uses some form of client credential to authenticate and generate a signed JWT token that is then passed back to the server on later requests.
>
> You may find it helpful to use your browser’s developer console while using the NiFi Web UI to look at the REST API network interactions between the UI and the server. This may show you how to obtain an authentication token (assuming that is how your NiFi instance is configured) and also how to perform operations such as start or stop a processor.
>
> Lastly, while this can be done using CURL, you may find it helpful to use one of the community clients, such as NiPyApi [2], which abstracts some of the details of interacting with the NiFi REST API.
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication>
> [2] https://github.com/Chaffelson/nipyapi <https://github.com/Chaffelson/nipyapi>
>
> From: Tian TD Deng <dengtian@cn.ibm.com <ma...@cn.ibm.com>>
> Reply-To: <users@nifi.apache.org <ma...@nifi.apache.org>>
> Date: Friday, March 16, 2018 at 11:20
> To: <users@nifi.apache.org <ma...@nifi.apache.org>>
> Cc: <users@nifi.apache.org <ma...@nifi.apache.org>>
> Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
>
> Hi Kevin,
>
> Thanks a lot for your help.
> I tried to modify the property " nifi.security.needClientAuth " from true to false, unfortunately, it didn't work, displayed the same error.
>
> However, I find that the "nifi.security.user.login.identity.provider" is not set to any value in the nifi.properties.
>
> So in this case, what should I do to send a HTTPS request to the Nifi Restful API to do something like start/stop the processor using curl?
>
> Thanks & Best Regards
>
> Joey(Tian) Deng
> -------------------------------------------------------------------------------------------
> IT Specialist
> IBM China Global Delivery Center
> Tel: (+86) 27 59269644
> E-mail: dengtian@cn.ibm.com <ma...@cn.ibm.com>
> WhatsApp: +86 15671158671
> -------------------------------------------------------------------------------------------
>
>
>> ----- Original message -----
>> From: Kevin Doran <kdoran@apache.org <ma...@apache.org>>
>> To: <users@nifi.apache.org <ma...@nifi.apache.org>>
>> Cc:
>> Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
>> Date: Fri, Mar 16, 2018 9:58 PM
>>
>>
>> When using access tokens, make sure your NiFi instance is configured to not require client certificates for the TLS certificates:
>>
>> nifi.security.needClientAuth=false
>>
>>
>> This is because you only want the NiFi server (not the client) to establish its identity with a TLS certificate; the client will establish identity after the TLS connection is established by passing the username and password credentials that will be validated by the Login Identity Provider you configured for NiFi. For example:
>>
>> nifi.security.user.login.identity.provider=ldap-provider
>>
>> where “ldap-provider” is configured in login-identity-providers.xml
>>
>> Hope this helps!
>> Kevin
>>
>> From: Tian TD Deng <dengtian@cn.ibm.com <ma...@cn.ibm.com>>
>> Reply-To: <users@nifi.apache.org <ma...@nifi.apache.org>>
>> Date: Friday, March 16, 2018 at 04:51
>> To: <users@nifi.apache.org <ma...@nifi.apache.org>>
>> Subject: Error when sending HTTPS request to Nifi using the Restful API
>>
>> Dear All, <>
>>
>> I was trying to send HTTPS request to Nifi using the Restful API via curl, however, it didn't work as expected.
>>
>> curl -k --tlsv1.2 https://localhost:8443/nifi-api/access/token <https://urldefense.proofpoint.com/v2/url?u=https-3A__localhost-3A8443_nifi-2Dapi_access_token&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5y4NPimmNGBw_uxhYgQmALaAwRfbIeRE6cMFQ9S3Shs&m=_lXJrQmuNdQxBiY5EymD-n9z3uR1tmWqLaoHGyRzU7Y&s=j8Ss_CTYXxglpLDJuNSzwH0COn30NbjKFylpMvzWeng&e=> --data 'username=test&password=password'
>>
>> <image001.png>
>>
>> Could you please help to give some advice about this?
>> Thanks a lot!
>>
>> Thanks & Best Regards
>>
>> Joey(Tian) Deng
>> -------------------------------------------------------------------------------------------
>> IT Specialist
>> IBM China Global Delivery Center
>> Tel: (+86) 27 59269644
>> E-mail: dengtian@cn.ibm.com <ma...@cn.ibm.com>
>> WhatsApp: +86 15671158671
>> -------------------------------------------------------------------------------------------
>>
>>
>>
>
>
>
> <image001.png>
Re: Error when sending HTTPS request to Nifi using the Restful API
Posted by Bryan Bende <bb...@gmail.com>.
To use a client certificate with curl you can do the following:
curl --cert-type P12 --cert <YOUR-P12-FILENAME>.p12:<PASSWORD> --cacert
nifi-cert.pem -v https://hostname:8443/nifi-api/process-groups
In this case nifi-cert.pem is the public key of the certificate authority
that was used to sign NiFi's server cert, which for my example came from
nifi-toolkit.
If your CA is not from nifi-toolkit then you'll need to figure out how to
get a PEM file for your CA's public key.
On Mon, Mar 19, 2018 at 2:38 AM, Tian TD Deng <de...@cn.ibm.com> wrote:
> Hi Kevim
>
> Thanks a lot for your explanation.
>
> The NiFi server is configured to authenticate clients using client
> certificates as below.
>
> Then I imported the .p12 certificate into Chrome, and I can invoke the
> Restful API now using Chrome.
>
> However, I couldn't do that using curl, so I was wondering should I add
> some parameters to involve the certificate to invoke the Restful API using
> curl?
> *Thanks & Best Regards*
>
> *Joey(Tian) Deng*
> ------------------------------------------------------------
> -------------------------------
> IT Specialist
> IBM China Global Delivery Center
> *Tel*: (+86) 27 59269644 <+86%2027%205926%209644>
> E-mail: dengtian@cn.ibm.com
> WhatsApp: +86 15671158671 <+86%20156%207115%208671>
> ------------------------------------------------------------
> -------------------------------
>
>
>
> ----- Original message -----
> From: Kevin Doran <kd...@apache.org>
> To: <us...@nifi.apache.org>
> Cc:
> Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
> Date: Fri, Mar 16, 2018 11:43 PM
>
>
> Any secured (HTTPS) NiFi instance requires client authentication for every
> REST API request, so the first thing you need to know is how your NiFi
> server is configured to authenticate clients. There are lots of options,
> including client certificates, login with LDAP username and password, or
> authenticate with Kerberos ticket, Knox identity, or OpenId connect. These
> options are explained in the NiFi Administration Guide [1]. One way to
> determine this is by looking at the files in the /conf directory (i.e.,
> nifi.properties).
>
>
>
> Once you know how the NiFi server authenticates client requests, you must
> configure the REST API client you are using to perform the same
> authentication steps. This varies based on authentication mechanism, but is
> basically either a client certificate provided during the TLS connection
> handshake or obtaining a JWT access token using one of the /access/* REST
> API endpoints that uses some form of client credential to authenticate and
> generate a signed JWT token that is then passed back to the server on later
> requests.
>
>
>
> You may find it helpful to use your browser’s developer console while
> using the NiFi Web UI to look at the REST API network interactions between
> the UI and the server. This may show you how to obtain an authentication
> token (assuming that is how your NiFi instance is configured) and also how
> to perform operations such as start or stop a processor.
>
>
>
> Lastly, while this can be done using CURL, you may find it helpful to use
> one of the community clients, such as NiPyApi [2], which abstracts some of
> the details of interacting with the NiFi REST API.
>
>
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-
> guide.html#user_authentication
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__nifi.apache.org_docs_nifi-2Ddocs_html_administration-2Dguide.html-23user-5Fauthentication&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5y4NPimmNGBw_uxhYgQmALaAwRfbIeRE6cMFQ9S3Shs&m=DW9ZmDuFkY_lPJHdbrmjmWmmX18T47cVKxeaFHve9XA&s=dcaINvaTMdVCrlDutK0YL8QIAgjcUHXwVqEQmmHAJBM&e=>
>
> [2] https://github.com/Chaffelson/nipyapi
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Chaffelson_nipyapi&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5y4NPimmNGBw_uxhYgQmALaAwRfbIeRE6cMFQ9S3Shs&m=DW9ZmDuFkY_lPJHdbrmjmWmmX18T47cVKxeaFHve9XA&s=-jn8WIiDoSI7I_5Y02-JqquuEO7KC_tzDhd355Nyyak&e=>
>
>
>
> *From: *Tian TD Deng <de...@cn.ibm.com>
> *Reply-To: *<us...@nifi.apache.org>
> *Date: *Friday, March 16, 2018 at 11:20
> *To: *<us...@nifi.apache.org>
> *Cc: *<us...@nifi.apache.org>
> *Subject: *Re: Error when sending HTTPS request to Nifi using the Restful
> API
>
>
>
> Hi Kevin,
>
>
>
> Thanks a lot for your help.
>
> I tried to modify the property " *nifi.security.needClientAuth* " from
> true to false, unfortunately, it didn't work, displayed the same error.
>
>
>
> However, I find that the "*nifi.security.user.login.identity.provider"*
> is not set to any value in the nifi.properties.
>
>
>
> So in this case, what should I do to send a *HTTPS* request to the Nifi
> Restful API to do something like start/stop the processor using *curl*?
>
>
>
> *Thanks & Best Regards*
>
>
> *Joey(Tian) Deng*
>
> ------------------------------------------------------------
> -------------------------------
> IT Specialist
> IBM China Global Delivery Center
> *Tel*: (+86) 27 59269644 <+86%2027%205926%209644>
> *E-mail: *dengtian@cn.ibm.com
>
> *WhatsApp:* +86 15671158671 <+86%20156%207115%208671>
> ------------------------------------------------------------
> -------------------------------
>
>
>
>
>
> ----- Original message -----
> From: Kevin Doran <kd...@apache.org>
> To: <us...@nifi.apache.org>
> Cc:
> Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
> Date: Fri, Mar 16, 2018 9:58 PM
>
>
> When using access tokens, make sure your NiFi instance is configured to
> not require client certificates for the TLS certificates:
>
>
>
> *nifi.security.needClientAuth*=false
>
>
>
> This is because you only want the NiFi *server* (not the client) to
> establish its identity with a TLS certificate; the client will establish
> identity after the TLS connection is established by passing the username
> and password credentials that will be validated by the Login Identity
> Provider you configured for NiFi. For example:
>
>
>
> *nifi.security.user.login.identity.provider*=*ldap-provider*
>
>
>
> where “ldap-provider” is configured in login-identity-providers.xml
>
>
>
> Hope this helps!
>
> Kevin
>
>
>
> *From: *Tian TD Deng <de...@cn.ibm.com>
> *Reply-To: *<us...@nifi.apache.org>
> *Date: *Friday, March 16, 2018 at 04:51
> *To: *<us...@nifi.apache.org>
> *Subject: *Error when sending HTTPS request to Nifi using the Restful API
>
>
>
> Dear All,
>
>
>
> I was trying to send HTTPS request to Nifi using the Restful API via curl,
> however, it didn't work as expected.
>
>
>
> curl -k --tlsv1.2 https://localhost:8443/nifi-api/access/token
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__localhost-3A8443_nifi-2Dapi_access_token&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=5y4NPimmNGBw_uxhYgQmALaAwRfbIeRE6cMFQ9S3Shs&m=_lXJrQmuNdQxBiY5EymD-n9z3uR1tmWqLaoHGyRzU7Y&s=j8Ss_CTYXxglpLDJuNSzwH0COn30NbjKFylpMvzWeng&e=>
> --data 'username=test&password=password'
>
>
>
> [image: cid:152117863809417]
>
>
>
> Could you please help to give some advice about this?
>
> Thanks a lot!
>
>
>
> *Thanks & Best Regards*
>
>
> *Joey(Tian) Deng*
>
> ------------------------------------------------------------
> -------------------------------
> IT Specialist
> IBM China Global Delivery Center
> *Tel*: (+86) 27 59269644 <+86%2027%205926%209644>
> *E-mail: *dengtian@cn.ibm.com
>
> *WhatsApp:* +86 15671158671 <+86%20156%207115%208671>
> ------------------------------------------------------------
> -------------------------------
>
>
>
>
>
>
>
>
>
>
>
Re: Error when sending HTTPS request to Nifi using the Restful API
Posted by Kevin Doran <kd...@apache.org>.
Any secured (HTTPS) NiFi instance requires client authentication for every REST API request, so the first thing you need to know is how your NiFi server is configured to authenticate clients. There are lots of options, including client certificates, login with LDAP username and password, or authenticate with Kerberos ticket, Knox identity, or OpenId connect. These options are explained in the NiFi Administration Guide [1]. One way to determine this is by looking at the files in the /conf directory (i.e., nifi.properties).
Once you know how the NiFi server authenticates client requests, you must configure the REST API client you are using to perform the same authentication steps. This varies based on authentication mechanism, but is basically either a client certificate provided during the TLS connection handshake or obtaining a JWT access token using one of the /access/* REST API endpoints that uses some form of client credential to authenticate and generate a signed JWT token that is then passed back to the server on later requests.
You may find it helpful to use your browser’s developer console while using the NiFi Web UI to look at the REST API network interactions between the UI and the server. This may show you how to obtain an authentication token (assuming that is how your NiFi instance is configured) and also how to perform operations such as start or stop a processor.
Lastly, while this can be done using CURL, you may find it helpful to use one of the community clients, such as NiPyApi [2], which abstracts some of the details of interacting with the NiFi REST API.
[1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication
[2] https://github.com/Chaffelson/nipyapi
From: Tian TD Deng <de...@cn.ibm.com>
Reply-To: <us...@nifi.apache.org>
Date: Friday, March 16, 2018 at 11:20
To: <us...@nifi.apache.org>
Cc: <us...@nifi.apache.org>
Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
Hi Kevin,
Thanks a lot for your help.
I tried to modify the property " nifi.security.needClientAuth " from true to false, unfortunately, it didn't work, displayed the same error.
However, I find that the "nifi.security.user.login.identity.provider" is not set to any value in the nifi.properties.
So in this case, what should I do to send a HTTPS request to the Nifi Restful API to do something like start/stop the processor using curl?
Thanks & Best Regards
Joey(Tian) Deng
-------------------------------------------------------------------------------------------
IT Specialist
IBM China Global Delivery Center
Tel: (+86) 27 59269644
E-mail: dengtian@cn.ibm.com
WhatsApp: +86 15671158671
-------------------------------------------------------------------------------------------
----- Original message -----
From: Kevin Doran <kd...@apache.org>
To: <us...@nifi.apache.org>
Cc:
Subject: Re: Error when sending HTTPS request to Nifi using the Restful API
Date: Fri, Mar 16, 2018 9:58 PM
When using access tokens, make sure your NiFi instance is configured to not require client certificates for the TLS certificates:
nifi.security.needClientAuth=false
This is because you only want the NiFi server (not the client) to establish its identity with a TLS certificate; the client will establish identity after the TLS connection is established by passing the username and password credentials that will be validated by the Login Identity Provider you configured for NiFi. For example:
nifi.security.user.login.identity.provider=ldap-provider
where “ldap-provider” is configured in login-identity-providers.xml
Hope this helps!
Kevin
From: Tian TD Deng <de...@cn.ibm.com>
Reply-To: <us...@nifi.apache.org>
Date: Friday, March 16, 2018 at 04:51
To: <us...@nifi.apache.org>
Subject: Error when sending HTTPS request to Nifi using the Restful API
Dear All,
I was trying to send HTTPS request to Nifi using the Restful API via curl, however, it didn't work as expected.
curl -k --tlsv1.2 https://localhost:8443/nifi-api/access/token --data 'username=test&password=password'
Could you please help to give some advice about this?
Thanks a lot!
Thanks & Best Regards
Joey(Tian) Deng
-------------------------------------------------------------------------------------------
IT Specialist
IBM China Global Delivery Center
Tel: (+86) 27 59269644
E-mail: dengtian@cn.ibm.com
WhatsApp: +86 15671158671
-------------------------------------------------------------------------------------------