You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Jon Logan <jm...@buffalo.edu> on 2018/10/12 16:42:35 UTC
Whitelisting Proxy Host values in a Container Environment?
We are running into issues with NiFi not allowing secure connections in a
container due to the proxy...the only documentation we've found on this
involves whitelisting specific proxy addresses. Is this the only solution?
Specifically, we're concerned about the fact that we don't know the proxy
address ahead of time to whitelist -- the port is an arbitrary assigned at
runtime port, and the proxy name could be any of the nodes of our
Kubernetes cluster.
Are we missing something?
Thanks!
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Andy LoPresto <al...@apache.org>.
No, ignoring the port could lead to the same malicious host injection in the scenario where multiple services are hosted on the same machine (i.e. nifi-1 at nifi.apache.org:8443 <http://nifi.apache.org:8443/> and nifi-2 at nifi.apache.org:8444). Is the port for each different than what is set nifi.web.https.port? That value should be automatically included.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Oct 19, 2018, at 12:13 AM, Jon Logan <jm...@buffalo.edu> wrote:
>
> Thanks Andy. Would a solution involving only checking the hostname and not the port be considered? That's where our heartache is coming from. The documentation hints that this is supported by referencing the format host[:port] but the port is not currently optional (unless using 80/443 I suppose).
>
> On Thu, Oct 18, 2018 at 4:58 AM Andy LoPresto <alopresto@apache.org <ma...@apache.org>> wrote:
> Jon,
>
> Sorry to hear that securing NiFi is proving difficult for you.
>
> The reason this feature was introduced is because prior to this check being enforced, a malicious Host header in requests could poison the server and return untrusted resources. Please see the Admin Guide [1] section on Proxy Configuration for more details. When not running behind a proxy, this feature is usually very simple to configure — the hostname of the NiFi service is automatically included in the whitelist, so requests are handled successfully. The original feature request (for proxy whitelisting) is in NIFI-4761 [2] (unit tests [3]) if you want more detail on why it was implemented, or see this article [4] for more details on host header attacks.
>
> If your dynamic hostnames fall in a list that is assigned on demand, you could provide the full list (comma-separated) as the whitelisted host value in all nodes, and this would allow each to be verified. If not, you could request a feature to allow regex-parsing of a pattern for that value as well, but I am opposed to this as it usually introduces more problems than it solves.
>
> I guess you could request an explicit disabling of this header check via nifi.properties, but I would certainly encourage you to find a solution that maintains the header check if possible.
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration>
> [2] https://issues.apache.org/jira/browse/NIFI-4761 <https://issues.apache.org/jira/browse/NIFI-4761>
> [3] https://github.com/apache/nifi/pull/2279/files <https://github.com/apache/nifi/pull/2279/files>
> [4] https://dzone.com/articles/what-is-a-host-header-attack <https://dzone.com/articles/what-is-a-host-header-attack>
>
>
>
>
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com <ma...@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>> On Oct 15, 2018, at 11:06 PM, Peter Wilcsinszky <peterwilcsinszky@gmail.com <ma...@gmail.com>> wrote:
>>
>> Jon,
>>
>> as for the port-forward you can bind NiFi to lo(required by the port-forward)+eth0 (I beleive eth0 is not provider dependent):
>> nifi.web.https.network.interface.lo=lo
>> nifi.web.https.network.interface.eth0=eth0
>>
>> Peter
>>
>> On Mon, Oct 15, 2018 at 3:46 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>> Thanks Peter. I briefly was playing around with connecting to localhost via the port-forwarding late last week but was having issues where Nifi was internally trying to connect to 0.0.0.0 and failing...I'll toy with it some more this morning (I had set the https listen property as 0.0.0.0 so that it'd bind to localhost). Or I think the NodePort option will work -- but will require mucking with the yaml file every deployment to avoid port clashing between deployments. Mildly inconvenient, but certainly is doable.
>>
>> If this feature were disabled, certificates would still work -- in this case, we know our entrypoint, but do not know the port. We could have certs dynamically issued including the entrypoint hostname into the cert as an alternate name. Our issue hasn't been knowing the hostname in advance, but knowing the port, which isn't part of the certificate entry. We haven't made it to that point yet, but it should work, and is on our roadmap.
>>
>> Again, thanks a lot -- securing this is turning out to be a lot more complicated than originally anticipated.
>>
>>
>>
>>
>> On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <peterwilcsinszky@gmail.com <ma...@gmail.com>> wrote:
>> Hey,
>>
>> I can't tell about the original intent and motivations, but this is the Jira that introduced this check [1].
>>
>> What I can tell is mutual TLS is not the only option to authenticate against NiFi. You can set up LDAP for example to authenticate the client and in that case MITM is possible I beleive.
>>
>> You will need a stable network identity that you can use to configure as your "proxy" in advance. For example in a testing scenario where you have access to the kubernetes cluster you can simply use "localhost" as the name of the proxy and use kubernetes port forward to tunnel requests from the localhost to your individual nodes (only one node at a time).
>>
>> Another option that could better work for non-local use cases is to use a LoadBalancer service in front of the nodes and configure DNS to point to your LoadBalancer IP. If you want to do this in advance it is possible to create floating IPs and preconfigure DNS for it at almost any cloud provider. Then add the configured DNS to nifi.web.proxy.host property when starting your cluster. If setting up DNS is not an option you can use the IP directly. If setting up the IP in advance is not an option you may use an arbitrary hostname as the proxy host and add that hostname to your hosts file (or dnsmasq or company dns) to point to the dynamically generated LoadBalancer IP after NiFi started up.
>>
>> Yet another option could be using Knox proxy that can act as an authentication gateway to your NiFi cluster. Knox can use a fully independent authentication scheme in front of the clients and at the same time authenticate against NiFi using NiFi's supported mechanisms. To get a grasp of how that works here is a detailed post on that matter [2].
>>
>> Maybe you try to connect to you NiFi nodes through Kubernetes provided NodePorts and want to access them directly. You can set up dedicated Kubernetes services if your node list is static but then you have to configure all the services with IP/DNS individually. Maybe you can leverage Knox here as well to route your request to the individual nodes.
>>
>> As for disabling this feature I'm not sure how that would work. You would still need to add the host as a SAN entry to your node certificates and that is something that must be set in advance. Of course you can decide to avoid TLS verification of the server, but then you cannot really trust the connection anymore. (You may think wildcard cert could be used there, but it is discouraged for several reasons)
>>
>> It would be good to know more about your general and most specifically about your security requirements to know what would be the easiest way to setup your cluster.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-4501 <https://issues.apache.org/jira/browse/NIFI-4501>
>> [2] https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html <https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html>
>>
>> Cheers,
>> Peter
>>
>> On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>> Thanks Jeff. We're not using Helm (I don't think at least!) and are using kubectl directly with yaml files. We are trying to avoid binding to explicit external ports so that we can deploy multiple times, and I am not sure how to determine the arbitrary external port assigned to be able to do the property substitution. Is there a way to do this or am I approaching this wrong?
>>
>> As an aside, I'm not really sure what this is accomplishing, as it seems like the 2-way SSL is preventing any sort of MITM attack. Is there a way to just turn off or bypass this check?
>>
>> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jtswork@gmail.com <ma...@gmail.com>> wrote:
>> Jon,
>>
>> I'll start off by saying that I'm still new to k8s, and don't know the specifics of most of this. You should be able to inject the host and port of the proxy into the NiFi configuration before starting the NiFi instances, with a Helm chart, for instance. I've done similar things with docker compose.
>>
>> You are correct, though, that nifi.web.proxy.host and nifi.web.proxy.context.path need to be set in nifi.properties before starting NiFi. This is required for security purposes, and allows NiFi return responses which are based on the proxy host and context path values so that references to resources hosted by NiFi are made through the proxy, instead of exposing the internal URI to a resource.
>>
>> - Jeff
>>
>> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>> We are running into issues with NiFi not allowing secure connections in a container due to the proxy...the only documentation we've found on this involves whitelisting specific proxy addresses. Is this the only solution? Specifically, we're concerned about the fact that we don't know the proxy address ahead of time to whitelist -- the port is an arbitrary assigned at runtime port, and the proxy name could be any of the nodes of our Kubernetes cluster.
>>
>> Are we missing something?
>>
>>
>> Thanks!
>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Jon Logan <jm...@buffalo.edu>.
Thanks Andy. Would a solution involving only checking the hostname and not
the port be considered? That's where our heartache is coming from. The
documentation hints that this is supported by referencing the
format host[:port] but the port is not currently optional (unless using
80/443 I suppose).
On Thu, Oct 18, 2018 at 4:58 AM Andy LoPresto <al...@apache.org> wrote:
> Jon,
>
> Sorry to hear that securing NiFi is proving difficult for you.
>
> The reason this feature was introduced is because prior to this check
> being enforced, a malicious Host header in requests could poison the server
> and return untrusted resources. Please see the Admin Guide [1] section on
> Proxy Configuration for more details. When not running behind a proxy, this
> feature is usually very simple to configure — the hostname of the NiFi
> service is automatically included in the whitelist, so requests are handled
> successfully. The original feature request (for proxy whitelisting) is in
> NIFI-4761 [2] (unit tests [3]) if you want more detail on why it was
> implemented, or see this article [4] for more details on host header
> attacks.
>
> If your dynamic hostnames fall in a list that is assigned on demand, you
> could provide the full list (comma-separated) as the whitelisted host value
> in all nodes, and this would allow each to be verified. If not, you could
> request a feature to allow regex-parsing of a pattern for that value as
> well, but I am opposed to this as it usually introduces more problems than
> it solves.
>
> I guess you could request an explicit disabling of this header check via
> nifi.properties, but I would certainly encourage you to find a solution
> that maintains the header check if possible.
>
> [1]
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration
> [2] https://issues.apache.org/jira/browse/NIFI-4761
> [3] https://github.com/apache/nifi/pull/2279/files
> [4] https://dzone.com/articles/what-is-a-host-header-attack
>
>
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Oct 15, 2018, at 11:06 PM, Peter Wilcsinszky <
> peterwilcsinszky@gmail.com> wrote:
>
> Jon,
>
> as for the port-forward you can bind NiFi to lo(required by the
> port-forward)+eth0 (I beleive eth0 is not provider dependent):
> nifi.web.https.network.interface.lo=lo
> nifi.web.https.network.interface.eth0=eth0
>
> Peter
>
> On Mon, Oct 15, 2018 at 3:46 PM Jon Logan <jm...@buffalo.edu> wrote:
>
>> Thanks Peter. I briefly was playing around with connecting to localhost
>> via the port-forwarding late last week but was having issues where Nifi was
>> internally trying to connect to 0.0.0.0 and failing...I'll toy with it some
>> more this morning (I had set the https listen property as 0.0.0.0 so that
>> it'd bind to localhost). Or I think the NodePort option will work -- but
>> will require mucking with the yaml file every deployment to avoid port
>> clashing between deployments. Mildly inconvenient, but certainly is doable.
>>
>> If this feature were disabled, certificates would still work -- in this
>> case, we know our entrypoint, but do not know the port. We could have certs
>> dynamically issued including the entrypoint hostname into the cert as an
>> alternate name. Our issue hasn't been knowing the hostname in advance, but
>> knowing the port, which isn't part of the certificate entry. We haven't
>> made it to that point yet, but it should work, and is on our roadmap.
>>
>> Again, thanks a lot -- securing this is turning out to be a lot more
>> complicated than originally anticipated.
>>
>>
>>
>>
>> On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <
>> peterwilcsinszky@gmail.com> wrote:
>>
>>> Hey,
>>>
>>> I can't tell about the original intent and motivations, but this is the
>>> Jira that introduced this check [1].
>>>
>>> What I can tell is mutual TLS is not the only option to authenticate
>>> against NiFi. You can set up LDAP for example to authenticate the client
>>> and in that case MITM is possible I beleive.
>>>
>>> You will need a stable network identity that you can use to configure as
>>> your "proxy" in advance. For example in a testing scenario where you have
>>> access to the kubernetes cluster you can simply use "localhost" as the name
>>> of the proxy and use kubernetes port forward to tunnel requests from the
>>> localhost to your individual nodes (only one node at a time).
>>>
>>> Another option that could better work for non-local use cases is to use
>>> a LoadBalancer service in front of the nodes and configure DNS to point to
>>> your LoadBalancer IP. If you want to do this in advance it is possible to
>>> create floating IPs and preconfigure DNS for it at almost any cloud
>>> provider. Then add the configured DNS to nifi.web.proxy.host property when
>>> starting your cluster. If setting up DNS is not an option you can use the
>>> IP directly. If setting up the IP in advance is not an option you may use
>>> an arbitrary hostname as the proxy host and add that hostname to your hosts
>>> file (or dnsmasq or company dns) to point to the dynamically generated
>>> LoadBalancer IP after NiFi started up.
>>>
>>> Yet another option could be using Knox proxy that can act as an
>>> authentication gateway to your NiFi cluster. Knox can use a fully
>>> independent authentication scheme in front of the clients and at the same
>>> time authenticate against NiFi using NiFi's supported mechanisms. To get a
>>> grasp of how that works here is a detailed post on that matter [2].
>>>
>>> Maybe you try to connect to you NiFi nodes through Kubernetes provided
>>> NodePorts and want to access them directly. You can set up dedicated
>>> Kubernetes services if your node list is static but then you have to
>>> configure all the services with IP/DNS individually. Maybe you can leverage
>>> Knox here as well to route your request to the individual nodes.
>>>
>>> As for disabling this feature I'm not sure how that would work. You
>>> would still need to add the host as a SAN entry to your node certificates
>>> and that is something that must be set in advance. Of course you can decide
>>> to avoid TLS verification of the server, but then you cannot really trust
>>> the connection anymore. (You may think wildcard cert could be used there,
>>> but it is discouraged for several reasons)
>>>
>>> It would be good to know more about your general and most specifically
>>> about your security requirements to know what would be the easiest way to
>>> setup your cluster.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-4501
>>> [2]
>>> https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html
>>>
>>> Cheers,
>>> Peter
>>>
>>> On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jm...@buffalo.edu> wrote:
>>>
>>>> Thanks Jeff. We're not using Helm (I don't think at least!) and are
>>>> using kubectl directly with yaml files. We are trying to avoid binding to
>>>> explicit external ports so that we can deploy multiple times, and I am not
>>>> sure how to determine the arbitrary external port assigned to be able to do
>>>> the property substitution. Is there a way to do this or am I approaching
>>>> this wrong?
>>>>
>>>> As an aside, I'm not really sure what this is accomplishing, as it
>>>> seems like the 2-way SSL is preventing any sort of MITM attack. Is there a
>>>> way to just turn off or bypass this check?
>>>>
>>>> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jt...@gmail.com> wrote:
>>>>
>>>>> Jon,
>>>>>
>>>>> I'll start off by saying that I'm still new to k8s, and don't know the
>>>>> specifics of most of this. You should be able to inject the host and port
>>>>> of the proxy into the NiFi configuration before starting the NiFi
>>>>> instances, with a Helm chart, for instance. I've done similar things with
>>>>> docker compose.
>>>>>
>>>>> You are correct, though, that nifi.web.proxy.host and
>>>>> nifi.web.proxy.context.path need to be set in nifi.properties before
>>>>> starting NiFi. This is required for security purposes, and allows NiFi
>>>>> return responses which are based on the proxy host and context path values
>>>>> so that references to resources hosted by NiFi are made through the proxy,
>>>>> instead of exposing the internal URI to a resource.
>>>>>
>>>>> - Jeff
>>>>>
>>>>> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu>
>>>>> wrote:
>>>>>
>>>>>> We are running into issues with NiFi not allowing secure connections
>>>>>> in a container due to the proxy...the only documentation we've found on
>>>>>> this involves whitelisting specific proxy addresses. Is this the only
>>>>>> solution? Specifically, we're concerned about the fact that we don't know
>>>>>> the proxy address ahead of time to whitelist -- the port is an arbitrary
>>>>>> assigned at runtime port, and the proxy name could be any of the nodes of
>>>>>> our Kubernetes cluster.
>>>>>>
>>>>>> Are we missing something?
>>>>>>
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>
>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Andy LoPresto <al...@apache.org>.
Jon,
Sorry to hear that securing NiFi is proving difficult for you.
The reason this feature was introduced is because prior to this check being enforced, a malicious Host header in requests could poison the server and return untrusted resources. Please see the Admin Guide [1] section on Proxy Configuration for more details. When not running behind a proxy, this feature is usually very simple to configure — the hostname of the NiFi service is automatically included in the whitelist, so requests are handled successfully. The original feature request (for proxy whitelisting) is in NIFI-4761 [2] (unit tests [3]) if you want more detail on why it was implemented, or see this article [4] for more details on host header attacks.
If your dynamic hostnames fall in a list that is assigned on demand, you could provide the full list (comma-separated) as the whitelisted host value in all nodes, and this would allow each to be verified. If not, you could request a feature to allow regex-parsing of a pattern for that value as well, but I am opposed to this as it usually introduces more problems than it solves.
I guess you could request an explicit disabling of this header check via nifi.properties, but I would certainly encourage you to find a solution that maintains the header check if possible.
[1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration>
[2] https://issues.apache.org/jira/browse/NIFI-4761 <https://issues.apache.org/jira/browse/NIFI-4761>
[3] https://github.com/apache/nifi/pull/2279/files <https://github.com/apache/nifi/pull/2279/files>
[4] https://dzone.com/articles/what-is-a-host-header-attack <https://dzone.com/articles/what-is-a-host-header-attack>
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Oct 15, 2018, at 11:06 PM, Peter Wilcsinszky <pe...@gmail.com> wrote:
>
> Jon,
>
> as for the port-forward you can bind NiFi to lo(required by the port-forward)+eth0 (I beleive eth0 is not provider dependent):
> nifi.web.https.network.interface.lo=lo
> nifi.web.https.network.interface.eth0=eth0
>
> Peter
>
> On Mon, Oct 15, 2018 at 3:46 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
> Thanks Peter. I briefly was playing around with connecting to localhost via the port-forwarding late last week but was having issues where Nifi was internally trying to connect to 0.0.0.0 and failing...I'll toy with it some more this morning (I had set the https listen property as 0.0.0.0 so that it'd bind to localhost). Or I think the NodePort option will work -- but will require mucking with the yaml file every deployment to avoid port clashing between deployments. Mildly inconvenient, but certainly is doable.
>
> If this feature were disabled, certificates would still work -- in this case, we know our entrypoint, but do not know the port. We could have certs dynamically issued including the entrypoint hostname into the cert as an alternate name. Our issue hasn't been knowing the hostname in advance, but knowing the port, which isn't part of the certificate entry. We haven't made it to that point yet, but it should work, and is on our roadmap.
>
> Again, thanks a lot -- securing this is turning out to be a lot more complicated than originally anticipated.
>
>
>
>
> On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <peterwilcsinszky@gmail.com <ma...@gmail.com>> wrote:
> Hey,
>
> I can't tell about the original intent and motivations, but this is the Jira that introduced this check [1].
>
> What I can tell is mutual TLS is not the only option to authenticate against NiFi. You can set up LDAP for example to authenticate the client and in that case MITM is possible I beleive.
>
> You will need a stable network identity that you can use to configure as your "proxy" in advance. For example in a testing scenario where you have access to the kubernetes cluster you can simply use "localhost" as the name of the proxy and use kubernetes port forward to tunnel requests from the localhost to your individual nodes (only one node at a time).
>
> Another option that could better work for non-local use cases is to use a LoadBalancer service in front of the nodes and configure DNS to point to your LoadBalancer IP. If you want to do this in advance it is possible to create floating IPs and preconfigure DNS for it at almost any cloud provider. Then add the configured DNS to nifi.web.proxy.host property when starting your cluster. If setting up DNS is not an option you can use the IP directly. If setting up the IP in advance is not an option you may use an arbitrary hostname as the proxy host and add that hostname to your hosts file (or dnsmasq or company dns) to point to the dynamically generated LoadBalancer IP after NiFi started up.
>
> Yet another option could be using Knox proxy that can act as an authentication gateway to your NiFi cluster. Knox can use a fully independent authentication scheme in front of the clients and at the same time authenticate against NiFi using NiFi's supported mechanisms. To get a grasp of how that works here is a detailed post on that matter [2].
>
> Maybe you try to connect to you NiFi nodes through Kubernetes provided NodePorts and want to access them directly. You can set up dedicated Kubernetes services if your node list is static but then you have to configure all the services with IP/DNS individually. Maybe you can leverage Knox here as well to route your request to the individual nodes.
>
> As for disabling this feature I'm not sure how that would work. You would still need to add the host as a SAN entry to your node certificates and that is something that must be set in advance. Of course you can decide to avoid TLS verification of the server, but then you cannot really trust the connection anymore. (You may think wildcard cert could be used there, but it is discouraged for several reasons)
>
> It would be good to know more about your general and most specifically about your security requirements to know what would be the easiest way to setup your cluster.
>
> [1] https://issues.apache.org/jira/browse/NIFI-4501 <https://issues.apache.org/jira/browse/NIFI-4501>
> [2] https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html <https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html>
>
> Cheers,
> Peter
>
> On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
> Thanks Jeff. We're not using Helm (I don't think at least!) and are using kubectl directly with yaml files. We are trying to avoid binding to explicit external ports so that we can deploy multiple times, and I am not sure how to determine the arbitrary external port assigned to be able to do the property substitution. Is there a way to do this or am I approaching this wrong?
>
> As an aside, I'm not really sure what this is accomplishing, as it seems like the 2-way SSL is preventing any sort of MITM attack. Is there a way to just turn off or bypass this check?
>
> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jtswork@gmail.com <ma...@gmail.com>> wrote:
> Jon,
>
> I'll start off by saying that I'm still new to k8s, and don't know the specifics of most of this. You should be able to inject the host and port of the proxy into the NiFi configuration before starting the NiFi instances, with a Helm chart, for instance. I've done similar things with docker compose.
>
> You are correct, though, that nifi.web.proxy.host and nifi.web.proxy.context.path need to be set in nifi.properties before starting NiFi. This is required for security purposes, and allows NiFi return responses which are based on the proxy host and context path values so that references to resources hosted by NiFi are made through the proxy, instead of exposing the internal URI to a resource.
>
> - Jeff
>
> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
> We are running into issues with NiFi not allowing secure connections in a container due to the proxy...the only documentation we've found on this involves whitelisting specific proxy addresses. Is this the only solution? Specifically, we're concerned about the fact that we don't know the proxy address ahead of time to whitelist -- the port is an arbitrary assigned at runtime port, and the proxy name could be any of the nodes of our Kubernetes cluster.
>
> Are we missing something?
>
>
> Thanks!
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Peter Wilcsinszky <pe...@gmail.com>.
Jon,
as for the port-forward you can bind NiFi to lo(required by the
port-forward)+eth0 (I beleive eth0 is not provider dependent):
nifi.web.https.network.interface.lo=lo
nifi.web.https.network.interface.eth0=eth0
Peter
On Mon, Oct 15, 2018 at 3:46 PM Jon Logan <jm...@buffalo.edu> wrote:
> Thanks Peter. I briefly was playing around with connecting to localhost
> via the port-forwarding late last week but was having issues where Nifi was
> internally trying to connect to 0.0.0.0 and failing...I'll toy with it some
> more this morning (I had set the https listen property as 0.0.0.0 so that
> it'd bind to localhost). Or I think the NodePort option will work -- but
> will require mucking with the yaml file every deployment to avoid port
> clashing between deployments. Mildly inconvenient, but certainly is doable.
>
> If this feature were disabled, certificates would still work -- in this
> case, we know our entrypoint, but do not know the port. We could have certs
> dynamically issued including the entrypoint hostname into the cert as an
> alternate name. Our issue hasn't been knowing the hostname in advance, but
> knowing the port, which isn't part of the certificate entry. We haven't
> made it to that point yet, but it should work, and is on our roadmap.
>
> Again, thanks a lot -- securing this is turning out to be a lot more
> complicated than originally anticipated.
>
>
>
>
> On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <
> peterwilcsinszky@gmail.com> wrote:
>
>> Hey,
>>
>> I can't tell about the original intent and motivations, but this is the
>> Jira that introduced this check [1].
>>
>> What I can tell is mutual TLS is not the only option to authenticate
>> against NiFi. You can set up LDAP for example to authenticate the client
>> and in that case MITM is possible I beleive.
>>
>> You will need a stable network identity that you can use to configure as
>> your "proxy" in advance. For example in a testing scenario where you have
>> access to the kubernetes cluster you can simply use "localhost" as the name
>> of the proxy and use kubernetes port forward to tunnel requests from the
>> localhost to your individual nodes (only one node at a time).
>>
>> Another option that could better work for non-local use cases is to use a
>> LoadBalancer service in front of the nodes and configure DNS to point to
>> your LoadBalancer IP. If you want to do this in advance it is possible to
>> create floating IPs and preconfigure DNS for it at almost any cloud
>> provider. Then add the configured DNS to nifi.web.proxy.host property when
>> starting your cluster. If setting up DNS is not an option you can use the
>> IP directly. If setting up the IP in advance is not an option you may use
>> an arbitrary hostname as the proxy host and add that hostname to your hosts
>> file (or dnsmasq or company dns) to point to the dynamically generated
>> LoadBalancer IP after NiFi started up.
>>
>> Yet another option could be using Knox proxy that can act as an
>> authentication gateway to your NiFi cluster. Knox can use a fully
>> independent authentication scheme in front of the clients and at the same
>> time authenticate against NiFi using NiFi's supported mechanisms. To get a
>> grasp of how that works here is a detailed post on that matter [2].
>>
>> Maybe you try to connect to you NiFi nodes through Kubernetes provided
>> NodePorts and want to access them directly. You can set up dedicated
>> Kubernetes services if your node list is static but then you have to
>> configure all the services with IP/DNS individually. Maybe you can leverage
>> Knox here as well to route your request to the individual nodes.
>>
>> As for disabling this feature I'm not sure how that would work. You would
>> still need to add the host as a SAN entry to your node certificates and
>> that is something that must be set in advance. Of course you can decide to
>> avoid TLS verification of the server, but then you cannot really trust the
>> connection anymore. (You may think wildcard cert could be used there, but
>> it is discouraged for several reasons)
>>
>> It would be good to know more about your general and most specifically
>> about your security requirements to know what would be the easiest way to
>> setup your cluster.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-4501
>> [2]
>> https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html
>>
>> Cheers,
>> Peter
>>
>> On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jm...@buffalo.edu> wrote:
>>
>>> Thanks Jeff. We're not using Helm (I don't think at least!) and are
>>> using kubectl directly with yaml files. We are trying to avoid binding to
>>> explicit external ports so that we can deploy multiple times, and I am not
>>> sure how to determine the arbitrary external port assigned to be able to do
>>> the property substitution. Is there a way to do this or am I approaching
>>> this wrong?
>>>
>>> As an aside, I'm not really sure what this is accomplishing, as it seems
>>> like the 2-way SSL is preventing any sort of MITM attack. Is there a way to
>>> just turn off or bypass this check?
>>>
>>> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jt...@gmail.com> wrote:
>>>
>>>> Jon,
>>>>
>>>> I'll start off by saying that I'm still new to k8s, and don't know the
>>>> specifics of most of this. You should be able to inject the host and port
>>>> of the proxy into the NiFi configuration before starting the NiFi
>>>> instances, with a Helm chart, for instance. I've done similar things with
>>>> docker compose.
>>>>
>>>> You are correct, though, that nifi.web.proxy.host and
>>>> nifi.web.proxy.context.path need to be set in nifi.properties before
>>>> starting NiFi. This is required for security purposes, and allows NiFi
>>>> return responses which are based on the proxy host and context path values
>>>> so that references to resources hosted by NiFi are made through the proxy,
>>>> instead of exposing the internal URI to a resource.
>>>>
>>>> - Jeff
>>>>
>>>> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu> wrote:
>>>>
>>>>> We are running into issues with NiFi not allowing secure connections
>>>>> in a container due to the proxy...the only documentation we've found on
>>>>> this involves whitelisting specific proxy addresses. Is this the only
>>>>> solution? Specifically, we're concerned about the fact that we don't know
>>>>> the proxy address ahead of time to whitelist -- the port is an arbitrary
>>>>> assigned at runtime port, and the proxy name could be any of the nodes of
>>>>> our Kubernetes cluster.
>>>>>
>>>>> Are we missing something?
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Jon Logan <jm...@buffalo.edu>.
Thanks Peter. I briefly was playing around with connecting to localhost via
the port-forwarding late last week but was having issues where Nifi was
internally trying to connect to 0.0.0.0 and failing...I'll toy with it some
more this morning (I had set the https listen property as 0.0.0.0 so that
it'd bind to localhost). Or I think the NodePort option will work -- but
will require mucking with the yaml file every deployment to avoid port
clashing between deployments. Mildly inconvenient, but certainly is doable.
If this feature were disabled, certificates would still work -- in this
case, we know our entrypoint, but do not know the port. We could have certs
dynamically issued including the entrypoint hostname into the cert as an
alternate name. Our issue hasn't been knowing the hostname in advance, but
knowing the port, which isn't part of the certificate entry. We haven't
made it to that point yet, but it should work, and is on our roadmap.
Again, thanks a lot -- securing this is turning out to be a lot more
complicated than originally anticipated.
On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <
peterwilcsinszky@gmail.com> wrote:
> Hey,
>
> I can't tell about the original intent and motivations, but this is the
> Jira that introduced this check [1].
>
> What I can tell is mutual TLS is not the only option to authenticate
> against NiFi. You can set up LDAP for example to authenticate the client
> and in that case MITM is possible I beleive.
>
> You will need a stable network identity that you can use to configure as
> your "proxy" in advance. For example in a testing scenario where you have
> access to the kubernetes cluster you can simply use "localhost" as the name
> of the proxy and use kubernetes port forward to tunnel requests from the
> localhost to your individual nodes (only one node at a time).
>
> Another option that could better work for non-local use cases is to use a
> LoadBalancer service in front of the nodes and configure DNS to point to
> your LoadBalancer IP. If you want to do this in advance it is possible to
> create floating IPs and preconfigure DNS for it at almost any cloud
> provider. Then add the configured DNS to nifi.web.proxy.host property when
> starting your cluster. If setting up DNS is not an option you can use the
> IP directly. If setting up the IP in advance is not an option you may use
> an arbitrary hostname as the proxy host and add that hostname to your hosts
> file (or dnsmasq or company dns) to point to the dynamically generated
> LoadBalancer IP after NiFi started up.
>
> Yet another option could be using Knox proxy that can act as an
> authentication gateway to your NiFi cluster. Knox can use a fully
> independent authentication scheme in front of the clients and at the same
> time authenticate against NiFi using NiFi's supported mechanisms. To get a
> grasp of how that works here is a detailed post on that matter [2].
>
> Maybe you try to connect to you NiFi nodes through Kubernetes provided
> NodePorts and want to access them directly. You can set up dedicated
> Kubernetes services if your node list is static but then you have to
> configure all the services with IP/DNS individually. Maybe you can leverage
> Knox here as well to route your request to the individual nodes.
>
> As for disabling this feature I'm not sure how that would work. You would
> still need to add the host as a SAN entry to your node certificates and
> that is something that must be set in advance. Of course you can decide to
> avoid TLS verification of the server, but then you cannot really trust the
> connection anymore. (You may think wildcard cert could be used there, but
> it is discouraged for several reasons)
>
> It would be good to know more about your general and most specifically
> about your security requirements to know what would be the easiest way to
> setup your cluster.
>
> [1] https://issues.apache.org/jira/browse/NIFI-4501
> [2]
> https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html
>
> Cheers,
> Peter
>
> On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jm...@buffalo.edu> wrote:
>
>> Thanks Jeff. We're not using Helm (I don't think at least!) and are using
>> kubectl directly with yaml files. We are trying to avoid binding to
>> explicit external ports so that we can deploy multiple times, and I am not
>> sure how to determine the arbitrary external port assigned to be able to do
>> the property substitution. Is there a way to do this or am I approaching
>> this wrong?
>>
>> As an aside, I'm not really sure what this is accomplishing, as it seems
>> like the 2-way SSL is preventing any sort of MITM attack. Is there a way to
>> just turn off or bypass this check?
>>
>> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jt...@gmail.com> wrote:
>>
>>> Jon,
>>>
>>> I'll start off by saying that I'm still new to k8s, and don't know the
>>> specifics of most of this. You should be able to inject the host and port
>>> of the proxy into the NiFi configuration before starting the NiFi
>>> instances, with a Helm chart, for instance. I've done similar things with
>>> docker compose.
>>>
>>> You are correct, though, that nifi.web.proxy.host and
>>> nifi.web.proxy.context.path need to be set in nifi.properties before
>>> starting NiFi. This is required for security purposes, and allows NiFi
>>> return responses which are based on the proxy host and context path values
>>> so that references to resources hosted by NiFi are made through the proxy,
>>> instead of exposing the internal URI to a resource.
>>>
>>> - Jeff
>>>
>>> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu> wrote:
>>>
>>>> We are running into issues with NiFi not allowing secure connections in
>>>> a container due to the proxy...the only documentation we've found on this
>>>> involves whitelisting specific proxy addresses. Is this the only solution?
>>>> Specifically, we're concerned about the fact that we don't know the proxy
>>>> address ahead of time to whitelist -- the port is an arbitrary assigned at
>>>> runtime port, and the proxy name could be any of the nodes of our
>>>> Kubernetes cluster.
>>>>
>>>> Are we missing something?
>>>>
>>>>
>>>> Thanks!
>>>>
>>>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Peter Wilcsinszky <pe...@gmail.com>.
Hey,
I can't tell about the original intent and motivations, but this is the
Jira that introduced this check [1].
What I can tell is mutual TLS is not the only option to authenticate
against NiFi. You can set up LDAP for example to authenticate the client
and in that case MITM is possible I beleive.
You will need a stable network identity that you can use to configure as
your "proxy" in advance. For example in a testing scenario where you have
access to the kubernetes cluster you can simply use "localhost" as the name
of the proxy and use kubernetes port forward to tunnel requests from the
localhost to your individual nodes (only one node at a time).
Another option that could better work for non-local use cases is to use a
LoadBalancer service in front of the nodes and configure DNS to point to
your LoadBalancer IP. If you want to do this in advance it is possible to
create floating IPs and preconfigure DNS for it at almost any cloud
provider. Then add the configured DNS to nifi.web.proxy.host property when
starting your cluster. If setting up DNS is not an option you can use the
IP directly. If setting up the IP in advance is not an option you may use
an arbitrary hostname as the proxy host and add that hostname to your hosts
file (or dnsmasq or company dns) to point to the dynamically generated
LoadBalancer IP after NiFi started up.
Yet another option could be using Knox proxy that can act as an
authentication gateway to your NiFi cluster. Knox can use a fully
independent authentication scheme in front of the clients and at the same
time authenticate against NiFi using NiFi's supported mechanisms. To get a
grasp of how that works here is a detailed post on that matter [2].
Maybe you try to connect to you NiFi nodes through Kubernetes provided
NodePorts and want to access them directly. You can set up dedicated
Kubernetes services if your node list is static but then you have to
configure all the services with IP/DNS individually. Maybe you can leverage
Knox here as well to route your request to the individual nodes.
As for disabling this feature I'm not sure how that would work. You would
still need to add the host as a SAN entry to your node certificates and
that is something that must be set in advance. Of course you can decide to
avoid TLS verification of the server, but then you cannot really trust the
connection anymore. (You may think wildcard cert could be used there, but
it is discouraged for several reasons)
It would be good to know more about your general and most specifically
about your security requirements to know what would be the easiest way to
setup your cluster.
[1] https://issues.apache.org/jira/browse/NIFI-4501
[2]
https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html
Cheers,
Peter
On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <jm...@buffalo.edu> wrote:
> Thanks Jeff. We're not using Helm (I don't think at least!) and are using
> kubectl directly with yaml files. We are trying to avoid binding to
> explicit external ports so that we can deploy multiple times, and I am not
> sure how to determine the arbitrary external port assigned to be able to do
> the property substitution. Is there a way to do this or am I approaching
> this wrong?
>
> As an aside, I'm not really sure what this is accomplishing, as it seems
> like the 2-way SSL is preventing any sort of MITM attack. Is there a way to
> just turn off or bypass this check?
>
> On Sun, Oct 14, 2018 at 4:33 PM Jeff <jt...@gmail.com> wrote:
>
>> Jon,
>>
>> I'll start off by saying that I'm still new to k8s, and don't know the
>> specifics of most of this. You should be able to inject the host and port
>> of the proxy into the NiFi configuration before starting the NiFi
>> instances, with a Helm chart, for instance. I've done similar things with
>> docker compose.
>>
>> You are correct, though, that nifi.web.proxy.host and
>> nifi.web.proxy.context.path need to be set in nifi.properties before
>> starting NiFi. This is required for security purposes, and allows NiFi
>> return responses which are based on the proxy host and context path values
>> so that references to resources hosted by NiFi are made through the proxy,
>> instead of exposing the internal URI to a resource.
>>
>> - Jeff
>>
>> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu> wrote:
>>
>>> We are running into issues with NiFi not allowing secure connections in
>>> a container due to the proxy...the only documentation we've found on this
>>> involves whitelisting specific proxy addresses. Is this the only solution?
>>> Specifically, we're concerned about the fact that we don't know the proxy
>>> address ahead of time to whitelist -- the port is an arbitrary assigned at
>>> runtime port, and the proxy name could be any of the nodes of our
>>> Kubernetes cluster.
>>>
>>> Are we missing something?
>>>
>>>
>>> Thanks!
>>>
>>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Jon Logan <jm...@buffalo.edu>.
Thanks Jeff. We're not using Helm (I don't think at least!) and are using
kubectl directly with yaml files. We are trying to avoid binding to
explicit external ports so that we can deploy multiple times, and I am not
sure how to determine the arbitrary external port assigned to be able to do
the property substitution. Is there a way to do this or am I approaching
this wrong?
As an aside, I'm not really sure what this is accomplishing, as it seems
like the 2-way SSL is preventing any sort of MITM attack. Is there a way to
just turn off or bypass this check?
On Sun, Oct 14, 2018 at 4:33 PM Jeff <jt...@gmail.com> wrote:
> Jon,
>
> I'll start off by saying that I'm still new to k8s, and don't know the
> specifics of most of this. You should be able to inject the host and port
> of the proxy into the NiFi configuration before starting the NiFi
> instances, with a Helm chart, for instance. I've done similar things with
> docker compose.
>
> You are correct, though, that nifi.web.proxy.host and
> nifi.web.proxy.context.path need to be set in nifi.properties before
> starting NiFi. This is required for security purposes, and allows NiFi
> return responses which are based on the proxy host and context path values
> so that references to resources hosted by NiFi are made through the proxy,
> instead of exposing the internal URI to a resource.
>
> - Jeff
>
> On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu> wrote:
>
>> We are running into issues with NiFi not allowing secure connections in a
>> container due to the proxy...the only documentation we've found on this
>> involves whitelisting specific proxy addresses. Is this the only solution?
>> Specifically, we're concerned about the fact that we don't know the proxy
>> address ahead of time to whitelist -- the port is an arbitrary assigned at
>> runtime port, and the proxy name could be any of the nodes of our
>> Kubernetes cluster.
>>
>> Are we missing something?
>>
>>
>> Thanks!
>>
>
Re: Whitelisting Proxy Host values in a Container Environment?
Posted by Jeff <jt...@gmail.com>.
Jon,
I'll start off by saying that I'm still new to k8s, and don't know the
specifics of most of this. You should be able to inject the host and port
of the proxy into the NiFi configuration before starting the NiFi
instances, with a Helm chart, for instance. I've done similar things with
docker compose.
You are correct, though, that nifi.web.proxy.host and
nifi.web.proxy.context.path need to be set in nifi.properties before
starting NiFi. This is required for security purposes, and allows NiFi
return responses which are based on the proxy host and context path values
so that references to resources hosted by NiFi are made through the proxy,
instead of exposing the internal URI to a resource.
- Jeff
On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <jm...@buffalo.edu> wrote:
> We are running into issues with NiFi not allowing secure connections in a
> container due to the proxy...the only documentation we've found on this
> involves whitelisting specific proxy addresses. Is this the only solution?
> Specifically, we're concerned about the fact that we don't know the proxy
> address ahead of time to whitelist -- the port is an arbitrary assigned at
> runtime port, and the proxy name could be any of the nodes of our
> Kubernetes cluster.
>
> Are we missing something?
>
>
> Thanks!
>