You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by Sandor Molnar <sm...@cloudera.com.INVALID> on 2019/02/25 14:33:49 UTC
[DISCUSS] Knox Shell command for downloading the public cert from a
Knox instance
Hi folks,
I've just started to think about how to resolve
https://issues.apache.org/jira/browse/KNOX-1418 and an approach could be:
1.) Server-side changes
I'm thinking of starting a new embedded Jetty instance when the gateway
server starts on a pre-configured port (e.g. 8100) with a simple HTTP
connector. This connector would have only one handler: Jetty's
ResourceHandler (
https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
).
Whenever the gateway is started we would export the public cert of the
gateway into a dedicated folder (e.g.
$GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We have
to configure the ResourceHandler to allow access to this folder only (thus
nothing else will be exposed through this new endpoint).
2.) KnoxShell-side changes
Within KnoxShell we should add a new command that simply hits the new
endpoint and save the output in the root of the current user (for instance
curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
~/gateway-client-trust.jks)
3.) Optionally, we may change KnoxCLI to inform the end-user about the new
location of the JKS certificate in case the user executes `knoxcli.sh
export-cert --type JKS` (it does not make sense to do the same what we
already have)
Any comments are highly appreciated!
I've already coded a POC and it works as expected:
$ rm -f ~/gateway-client-trust.jks
$ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
~/gateway-client-trust.jks
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 674 100 674 0 0 43602 0 --:--:-- --:--:-- --:--:--
44933
$ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
Enter username: guest
Enter password:
[app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
Thanks,
Sandor
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by Sandeep Moré <mo...@gmail.com>.
I agree with Kevin, having to endpoints would be a bit confusing for the
users (HTTP for this and HTTPS for others) as well as from the code
perspective, also I am not sure how resource intensive two jetty servers
would be, I hear that it is not that bad.
On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org> wrote:
> I'm not a fan of adding a new endpoint. This port would need to be
> configurable. We would need to ensure that it always points to the correct
> location.
>
> Instead, can we download the cert from the existing HTTPS endpoint? We
> would have to not trust the TLS connection to pull the public cert, but
> this is the same as pulling the truststore over the HTTP endpoint you
> proposed. This will guarantee we don't have to do anything special on
> startup and will work with any configured TLS certificate.
>
> Kevin Risden
>
>
> On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar <smolnar@cloudera.com.invalid
> >
> wrote:
>
> > Hi folks,
> >
> > I've just started to think about how to resolve
> > https://issues.apache.org/jira/browse/KNOX-1418 and an approach could
> be:
> >
> > 1.) Server-side changes
> > I'm thinking of starting a new embedded Jetty instance when the gateway
> > server starts on a pre-configured port (e.g. 8100) with a simple HTTP
> > connector. This connector would have only one handler: Jetty's
> > ResourceHandler (
> >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > ).
> > Whenever the gateway is started we would export the public cert of the
> > gateway into a dedicated folder (e.g.
> > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We have
> > to configure the ResourceHandler to allow access to this folder only
> (thus
> > nothing else will be exposed through this new endpoint).
> >
> > 2.) KnoxShell-side changes
> > Within KnoxShell we should add a new command that simply hits the new
> > endpoint and save the output in the root of the current user (for
> instance
> > curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > ~/gateway-client-trust.jks)
> >
> > 3.) Optionally, we may change KnoxCLI to inform the end-user about the
> new
> > location of the JKS certificate in case the user executes `knoxcli.sh
> > export-cert --type JKS` (it does not make sense to do the same what we
> > already have)
> >
> > Any comments are highly appreciated!
> >
> > I've already coded a POC and it works as expected:
> >
> > $ rm -f ~/gateway-client-trust.jks
> >
> > $ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > ~/gateway-client-trust.jks
> > % Total % Received % Xferd Average Speed Time Time Time
> > Current
> > Dload Upload Total Spent Left
> > Speed
> > 100 674 100 674 0 0 43602 0 --:--:-- --:--:-- --:--:--
> > 44933
> >
> > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > Enter username: guest
> > Enter password:
> > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> >
> >
> > Thanks,
> > Sandor
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by Sandor Molnar <sm...@cloudera.com.INVALID>.
As pointed out by Kevin the KnoxShell can download the cert today by just
going to https://KNOX_HOSTNAME:KNOX_PORT. The only thing we need to
implement within the scope of this JIRA is to add a new KnoxShell
command to make it happen.
Larry's recommendation about creating a new landing page that has links to
admin UI, PEM and JKS truststores, etc... should be covered later on; new
JIRAs should be created and addressed.
On Mon, Feb 25, 2019 at 5:29 PM larry mccay <lm...@apache.org> wrote:
> I think it is mostly a client side replacement for what KnoxCLI export-cert
> does which doesn't require SSH to the gateway machine.
> Think about KnoxShell users and that they may only have line of site of the
> gateway endpoints but not access to the machine.
>
> Yes, one could do the same with openssl s_client but we could do better
> than that for them.
>
> On Mon, Feb 25, 2019 at 11:13 AM Robert Levas <rlevas@cloudera.com.invalid
> >
> wrote:
>
> > What is the use case for this?
> >
> > If a user just wants to download the TLS certificate, couldn't they
> execute
> > the following on the command line?
> >
> > openssl s_client -connect *knoxhost*:*knoxport* -showcerts </dev/null
> > 2>/dev/null | openssl x509 -outform PEM > knox_gateway.pem
> >
> >
> > Then the user can import the PEM file into their truststore.
> >
> > Or maybe the KnoxCLI can do this for the user.
> >
> >
> > On Mon, Feb 25, 2019 at 10:48 AM Phil Zampino <pz...@apache.org>
> wrote:
> >
> > > I'll echo the reservations around the overhead (config, performance,
> > > etc...) associated with an additional endpoint.
> > >
> > > I like the idea of leveraging the browser (cited by Larry), which
> already
> > > has a built-in mechanism for allowing the user to explicitly allow the
> > > interaction with the potentially untrusted Knox endpoint (e.g.,
> > self-signed
> > > cert deployment).
> > >
> > > The solution might be as simple as a download link on the Admin UI. The
> > > KnoxShell command could use loginInsecure() to access that link
> location,
> > > presuming it will be a well-known resource path.
> > >
> > > Thanks,
> > > Phil
> > >
> > >
> > > On Mon, Feb 25, 2019 at 10:06 AM larry mccay <lm...@apache.org>
> wrote:
> > >
> > > > Hi Sandor -
> > > >
> > > > Thanks for starting this discussion and taking up that task!
> > > >
> > > > +1 to Kevin's points.
> > > > KnoxSession already has a loginInsecure() method as well.
> > > >
> > > > I also think that it needs to be available from the Admin UI - in
> which
> > > > case the SSL cert is already trusted by your browser or the exception
> > > > already added.
> > > > The API use from KnoxShell is a bit odd as there is no protection
> > > against a
> > > > MITM returning a cert and intercepting all KnoxShell requests.
> > > > I'd like the primary mechanism to be a download link.
> > > > At least when the cert is CA signed, the browser will validate the
> > > hostname
> > > > and that it is trusted as part of the download whereas the
> > > loginInsecure()
> > > > will not.
> > > > Perhaps, we should try login() first and when it fails hostname
> > > validation
> > > > then we fallback to loginInsecure().
> > > >
> > > > thanks,
> > > >
> > > > --larry
> > > >
> > > > On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org>
> > wrote:
> > > >
> > > > > I'm not a fan of adding a new endpoint. This port would need to be
> > > > > configurable. We would need to ensure that it always points to the
> > > > correct
> > > > > location.
> > > > >
> > > > > Instead, can we download the cert from the existing HTTPS endpoint?
> > We
> > > > > would have to not trust the TLS connection to pull the public cert,
> > but
> > > > > this is the same as pulling the truststore over the HTTP endpoint
> you
> > > > > proposed. This will guarantee we don't have to do anything special
> on
> > > > > startup and will work with any configured TLS certificate.
> > > > >
> > > > > Kevin Risden
> > > > >
> > > > >
> > > > > On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar
> > > > <smolnar@cloudera.com.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > I've just started to think about how to resolve
> > > > > > https://issues.apache.org/jira/browse/KNOX-1418 and an approach
> > > could
> > > > > be:
> > > > > >
> > > > > > 1.) Server-side changes
> > > > > > I'm thinking of starting a new embedded Jetty instance when the
> > > gateway
> > > > > > server starts on a pre-configured port (e.g. 8100) with a simple
> > HTTP
> > > > > > connector. This connector would have only one handler: Jetty's
> > > > > > ResourceHandler (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > > > > > ).
> > > > > > Whenever the gateway is started we would export the public cert
> of
> > > the
> > > > > > gateway into a dedicated folder (e.g.
> > > > > > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks).
> > We
> > > > have
> > > > > > to configure the ResourceHandler to allow access to this folder
> > only
> > > > > (thus
> > > > > > nothing else will be exposed through this new endpoint).
> > > > > >
> > > > > > 2.) KnoxShell-side changes
> > > > > > Within KnoxShell we should add a new command that simply hits the
> > new
> > > > > > endpoint and save the output in the root of the current user (for
> > > > > instance
> > > > > > curl
> http://c7401.ambari.apache.org:8100/gateway-client-trust.jks
> > >
> > > > > > ~/gateway-client-trust.jks)
> > > > > >
> > > > > > 3.) Optionally, we may change KnoxCLI to inform the end-user
> about
> > > the
> > > > > new
> > > > > > location of the JKS certificate in case the user executes
> > `knoxcli.sh
> > > > > > export-cert --type JKS` (it does not make sense to do the same
> what
> > > we
> > > > > > already have)
> > > > > >
> > > > > > Any comments are highly appreciated!
> > > > > >
> > > > > > I've already coded a POC and it works as expected:
> > > > > >
> > > > > > $ rm -f ~/gateway-client-trust.jks
> > > > > >
> > > > > > $ curl
> > http://c7401.ambari.apache.org:8100/gateway-client-trust.jks
> > > >
> > > > > > ~/gateway-client-trust.jks
> > > > > > % Total % Received % Xferd Average Speed Time Time
> > > Time
> > > > > > Current
> > > > > > Dload Upload Total Spent
> > > Left
> > > > > > Speed
> > > > > > 100 674 100 674 0 0 43602 0 --:--:-- --:--:--
> > > > --:--:--
> > > > > > 44933
> > > > > >
> > > > > > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > > > > > Enter username: guest
> > > > > > Enter password:
> > > > > > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Sandor
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by larry mccay <lm...@apache.org>.
I think it is mostly a client side replacement for what KnoxCLI export-cert
does which doesn't require SSH to the gateway machine.
Think about KnoxShell users and that they may only have line of site of the
gateway endpoints but not access to the machine.
Yes, one could do the same with openssl s_client but we could do better
than that for them.
On Mon, Feb 25, 2019 at 11:13 AM Robert Levas <rl...@cloudera.com.invalid>
wrote:
> What is the use case for this?
>
> If a user just wants to download the TLS certificate, couldn't they execute
> the following on the command line?
>
> openssl s_client -connect *knoxhost*:*knoxport* -showcerts </dev/null
> 2>/dev/null | openssl x509 -outform PEM > knox_gateway.pem
>
>
> Then the user can import the PEM file into their truststore.
>
> Or maybe the KnoxCLI can do this for the user.
>
>
> On Mon, Feb 25, 2019 at 10:48 AM Phil Zampino <pz...@apache.org> wrote:
>
> > I'll echo the reservations around the overhead (config, performance,
> > etc...) associated with an additional endpoint.
> >
> > I like the idea of leveraging the browser (cited by Larry), which already
> > has a built-in mechanism for allowing the user to explicitly allow the
> > interaction with the potentially untrusted Knox endpoint (e.g.,
> self-signed
> > cert deployment).
> >
> > The solution might be as simple as a download link on the Admin UI. The
> > KnoxShell command could use loginInsecure() to access that link location,
> > presuming it will be a well-known resource path.
> >
> > Thanks,
> > Phil
> >
> >
> > On Mon, Feb 25, 2019 at 10:06 AM larry mccay <lm...@apache.org> wrote:
> >
> > > Hi Sandor -
> > >
> > > Thanks for starting this discussion and taking up that task!
> > >
> > > +1 to Kevin's points.
> > > KnoxSession already has a loginInsecure() method as well.
> > >
> > > I also think that it needs to be available from the Admin UI - in which
> > > case the SSL cert is already trusted by your browser or the exception
> > > already added.
> > > The API use from KnoxShell is a bit odd as there is no protection
> > against a
> > > MITM returning a cert and intercepting all KnoxShell requests.
> > > I'd like the primary mechanism to be a download link.
> > > At least when the cert is CA signed, the browser will validate the
> > hostname
> > > and that it is trusted as part of the download whereas the
> > loginInsecure()
> > > will not.
> > > Perhaps, we should try login() first and when it fails hostname
> > validation
> > > then we fallback to loginInsecure().
> > >
> > > thanks,
> > >
> > > --larry
> > >
> > > On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org>
> wrote:
> > >
> > > > I'm not a fan of adding a new endpoint. This port would need to be
> > > > configurable. We would need to ensure that it always points to the
> > > correct
> > > > location.
> > > >
> > > > Instead, can we download the cert from the existing HTTPS endpoint?
> We
> > > > would have to not trust the TLS connection to pull the public cert,
> but
> > > > this is the same as pulling the truststore over the HTTP endpoint you
> > > > proposed. This will guarantee we don't have to do anything special on
> > > > startup and will work with any configured TLS certificate.
> > > >
> > > > Kevin Risden
> > > >
> > > >
> > > > On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar
> > > <smolnar@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > I've just started to think about how to resolve
> > > > > https://issues.apache.org/jira/browse/KNOX-1418 and an approach
> > could
> > > > be:
> > > > >
> > > > > 1.) Server-side changes
> > > > > I'm thinking of starting a new embedded Jetty instance when the
> > gateway
> > > > > server starts on a pre-configured port (e.g. 8100) with a simple
> HTTP
> > > > > connector. This connector would have only one handler: Jetty's
> > > > > ResourceHandler (
> > > > >
> > > > >
> > > >
> > >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > > > > ).
> > > > > Whenever the gateway is started we would export the public cert of
> > the
> > > > > gateway into a dedicated folder (e.g.
> > > > > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks).
> We
> > > have
> > > > > to configure the ResourceHandler to allow access to this folder
> only
> > > > (thus
> > > > > nothing else will be exposed through this new endpoint).
> > > > >
> > > > > 2.) KnoxShell-side changes
> > > > > Within KnoxShell we should add a new command that simply hits the
> new
> > > > > endpoint and save the output in the root of the current user (for
> > > > instance
> > > > > curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks
> >
> > > > > ~/gateway-client-trust.jks)
> > > > >
> > > > > 3.) Optionally, we may change KnoxCLI to inform the end-user about
> > the
> > > > new
> > > > > location of the JKS certificate in case the user executes
> `knoxcli.sh
> > > > > export-cert --type JKS` (it does not make sense to do the same what
> > we
> > > > > already have)
> > > > >
> > > > > Any comments are highly appreciated!
> > > > >
> > > > > I've already coded a POC and it works as expected:
> > > > >
> > > > > $ rm -f ~/gateway-client-trust.jks
> > > > >
> > > > > $ curl
> http://c7401.ambari.apache.org:8100/gateway-client-trust.jks
> > >
> > > > > ~/gateway-client-trust.jks
> > > > > % Total % Received % Xferd Average Speed Time Time
> > Time
> > > > > Current
> > > > > Dload Upload Total Spent
> > Left
> > > > > Speed
> > > > > 100 674 100 674 0 0 43602 0 --:--:-- --:--:--
> > > --:--:--
> > > > > 44933
> > > > >
> > > > > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > > > > Enter username: guest
> > > > > Enter password:
> > > > > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Sandor
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by Robert Levas <rl...@cloudera.com.INVALID>.
What is the use case for this?
If a user just wants to download the TLS certificate, couldn't they execute
the following on the command line?
openssl s_client -connect *knoxhost*:*knoxport* -showcerts </dev/null
2>/dev/null | openssl x509 -outform PEM > knox_gateway.pem
Then the user can import the PEM file into their truststore.
Or maybe the KnoxCLI can do this for the user.
On Mon, Feb 25, 2019 at 10:48 AM Phil Zampino <pz...@apache.org> wrote:
> I'll echo the reservations around the overhead (config, performance,
> etc...) associated with an additional endpoint.
>
> I like the idea of leveraging the browser (cited by Larry), which already
> has a built-in mechanism for allowing the user to explicitly allow the
> interaction with the potentially untrusted Knox endpoint (e.g., self-signed
> cert deployment).
>
> The solution might be as simple as a download link on the Admin UI. The
> KnoxShell command could use loginInsecure() to access that link location,
> presuming it will be a well-known resource path.
>
> Thanks,
> Phil
>
>
> On Mon, Feb 25, 2019 at 10:06 AM larry mccay <lm...@apache.org> wrote:
>
> > Hi Sandor -
> >
> > Thanks for starting this discussion and taking up that task!
> >
> > +1 to Kevin's points.
> > KnoxSession already has a loginInsecure() method as well.
> >
> > I also think that it needs to be available from the Admin UI - in which
> > case the SSL cert is already trusted by your browser or the exception
> > already added.
> > The API use from KnoxShell is a bit odd as there is no protection
> against a
> > MITM returning a cert and intercepting all KnoxShell requests.
> > I'd like the primary mechanism to be a download link.
> > At least when the cert is CA signed, the browser will validate the
> hostname
> > and that it is trusted as part of the download whereas the
> loginInsecure()
> > will not.
> > Perhaps, we should try login() first and when it fails hostname
> validation
> > then we fallback to loginInsecure().
> >
> > thanks,
> >
> > --larry
> >
> > On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org> wrote:
> >
> > > I'm not a fan of adding a new endpoint. This port would need to be
> > > configurable. We would need to ensure that it always points to the
> > correct
> > > location.
> > >
> > > Instead, can we download the cert from the existing HTTPS endpoint? We
> > > would have to not trust the TLS connection to pull the public cert, but
> > > this is the same as pulling the truststore over the HTTP endpoint you
> > > proposed. This will guarantee we don't have to do anything special on
> > > startup and will work with any configured TLS certificate.
> > >
> > > Kevin Risden
> > >
> > >
> > > On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar
> > <smolnar@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I've just started to think about how to resolve
> > > > https://issues.apache.org/jira/browse/KNOX-1418 and an approach
> could
> > > be:
> > > >
> > > > 1.) Server-side changes
> > > > I'm thinking of starting a new embedded Jetty instance when the
> gateway
> > > > server starts on a pre-configured port (e.g. 8100) with a simple HTTP
> > > > connector. This connector would have only one handler: Jetty's
> > > > ResourceHandler (
> > > >
> > > >
> > >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > > > ).
> > > > Whenever the gateway is started we would export the public cert of
> the
> > > > gateway into a dedicated folder (e.g.
> > > > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We
> > have
> > > > to configure the ResourceHandler to allow access to this folder only
> > > (thus
> > > > nothing else will be exposed through this new endpoint).
> > > >
> > > > 2.) KnoxShell-side changes
> > > > Within KnoxShell we should add a new command that simply hits the new
> > > > endpoint and save the output in the root of the current user (for
> > > instance
> > > > curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > > > ~/gateway-client-trust.jks)
> > > >
> > > > 3.) Optionally, we may change KnoxCLI to inform the end-user about
> the
> > > new
> > > > location of the JKS certificate in case the user executes `knoxcli.sh
> > > > export-cert --type JKS` (it does not make sense to do the same what
> we
> > > > already have)
> > > >
> > > > Any comments are highly appreciated!
> > > >
> > > > I've already coded a POC and it works as expected:
> > > >
> > > > $ rm -f ~/gateway-client-trust.jks
> > > >
> > > > $ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks
> >
> > > > ~/gateway-client-trust.jks
> > > > % Total % Received % Xferd Average Speed Time Time
> Time
> > > > Current
> > > > Dload Upload Total Spent
> Left
> > > > Speed
> > > > 100 674 100 674 0 0 43602 0 --:--:-- --:--:--
> > --:--:--
> > > > 44933
> > > >
> > > > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > > > Enter username: guest
> > > > Enter password:
> > > > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> > > >
> > > >
> > > > Thanks,
> > > > Sandor
> > > >
> > >
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by Phil Zampino <pz...@apache.org>.
I'll echo the reservations around the overhead (config, performance,
etc...) associated with an additional endpoint.
I like the idea of leveraging the browser (cited by Larry), which already
has a built-in mechanism for allowing the user to explicitly allow the
interaction with the potentially untrusted Knox endpoint (e.g., self-signed
cert deployment).
The solution might be as simple as a download link on the Admin UI. The
KnoxShell command could use loginInsecure() to access that link location,
presuming it will be a well-known resource path.
Thanks,
Phil
On Mon, Feb 25, 2019 at 10:06 AM larry mccay <lm...@apache.org> wrote:
> Hi Sandor -
>
> Thanks for starting this discussion and taking up that task!
>
> +1 to Kevin's points.
> KnoxSession already has a loginInsecure() method as well.
>
> I also think that it needs to be available from the Admin UI - in which
> case the SSL cert is already trusted by your browser or the exception
> already added.
> The API use from KnoxShell is a bit odd as there is no protection against a
> MITM returning a cert and intercepting all KnoxShell requests.
> I'd like the primary mechanism to be a download link.
> At least when the cert is CA signed, the browser will validate the hostname
> and that it is trusted as part of the download whereas the loginInsecure()
> will not.
> Perhaps, we should try login() first and when it fails hostname validation
> then we fallback to loginInsecure().
>
> thanks,
>
> --larry
>
> On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org> wrote:
>
> > I'm not a fan of adding a new endpoint. This port would need to be
> > configurable. We would need to ensure that it always points to the
> correct
> > location.
> >
> > Instead, can we download the cert from the existing HTTPS endpoint? We
> > would have to not trust the TLS connection to pull the public cert, but
> > this is the same as pulling the truststore over the HTTP endpoint you
> > proposed. This will guarantee we don't have to do anything special on
> > startup and will work with any configured TLS certificate.
> >
> > Kevin Risden
> >
> >
> > On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar
> <smolnar@cloudera.com.invalid
> > >
> > wrote:
> >
> > > Hi folks,
> > >
> > > I've just started to think about how to resolve
> > > https://issues.apache.org/jira/browse/KNOX-1418 and an approach could
> > be:
> > >
> > > 1.) Server-side changes
> > > I'm thinking of starting a new embedded Jetty instance when the gateway
> > > server starts on a pre-configured port (e.g. 8100) with a simple HTTP
> > > connector. This connector would have only one handler: Jetty's
> > > ResourceHandler (
> > >
> > >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > > ).
> > > Whenever the gateway is started we would export the public cert of the
> > > gateway into a dedicated folder (e.g.
> > > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We
> have
> > > to configure the ResourceHandler to allow access to this folder only
> > (thus
> > > nothing else will be exposed through this new endpoint).
> > >
> > > 2.) KnoxShell-side changes
> > > Within KnoxShell we should add a new command that simply hits the new
> > > endpoint and save the output in the root of the current user (for
> > instance
> > > curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > > ~/gateway-client-trust.jks)
> > >
> > > 3.) Optionally, we may change KnoxCLI to inform the end-user about the
> > new
> > > location of the JKS certificate in case the user executes `knoxcli.sh
> > > export-cert --type JKS` (it does not make sense to do the same what we
> > > already have)
> > >
> > > Any comments are highly appreciated!
> > >
> > > I've already coded a POC and it works as expected:
> > >
> > > $ rm -f ~/gateway-client-trust.jks
> > >
> > > $ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > > ~/gateway-client-trust.jks
> > > % Total % Received % Xferd Average Speed Time Time Time
> > > Current
> > > Dload Upload Total Spent Left
> > > Speed
> > > 100 674 100 674 0 0 43602 0 --:--:-- --:--:--
> --:--:--
> > > 44933
> > >
> > > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > > Enter username: guest
> > > Enter password:
> > > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> > >
> > >
> > > Thanks,
> > > Sandor
> > >
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by larry mccay <lm...@apache.org>.
Hi Sandor -
Thanks for starting this discussion and taking up that task!
+1 to Kevin's points.
KnoxSession already has a loginInsecure() method as well.
I also think that it needs to be available from the Admin UI - in which
case the SSL cert is already trusted by your browser or the exception
already added.
The API use from KnoxShell is a bit odd as there is no protection against a
MITM returning a cert and intercepting all KnoxShell requests.
I'd like the primary mechanism to be a download link.
At least when the cert is CA signed, the browser will validate the hostname
and that it is trusted as part of the download whereas the loginInsecure()
will not.
Perhaps, we should try login() first and when it fails hostname validation
then we fallback to loginInsecure().
thanks,
--larry
On Mon, Feb 25, 2019 at 9:54 AM Kevin Risden <kr...@apache.org> wrote:
> I'm not a fan of adding a new endpoint. This port would need to be
> configurable. We would need to ensure that it always points to the correct
> location.
>
> Instead, can we download the cert from the existing HTTPS endpoint? We
> would have to not trust the TLS connection to pull the public cert, but
> this is the same as pulling the truststore over the HTTP endpoint you
> proposed. This will guarantee we don't have to do anything special on
> startup and will work with any configured TLS certificate.
>
> Kevin Risden
>
>
> On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar <smolnar@cloudera.com.invalid
> >
> wrote:
>
> > Hi folks,
> >
> > I've just started to think about how to resolve
> > https://issues.apache.org/jira/browse/KNOX-1418 and an approach could
> be:
> >
> > 1.) Server-side changes
> > I'm thinking of starting a new embedded Jetty instance when the gateway
> > server starts on a pre-configured port (e.g. 8100) with a simple HTTP
> > connector. This connector would have only one handler: Jetty's
> > ResourceHandler (
> >
> >
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> > ).
> > Whenever the gateway is started we would export the public cert of the
> > gateway into a dedicated folder (e.g.
> > $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We have
> > to configure the ResourceHandler to allow access to this folder only
> (thus
> > nothing else will be exposed through this new endpoint).
> >
> > 2.) KnoxShell-side changes
> > Within KnoxShell we should add a new command that simply hits the new
> > endpoint and save the output in the root of the current user (for
> instance
> > curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > ~/gateway-client-trust.jks)
> >
> > 3.) Optionally, we may change KnoxCLI to inform the end-user about the
> new
> > location of the JKS certificate in case the user executes `knoxcli.sh
> > export-cert --type JKS` (it does not make sense to do the same what we
> > already have)
> >
> > Any comments are highly appreciated!
> >
> > I've already coded a POC and it works as expected:
> >
> > $ rm -f ~/gateway-client-trust.jks
> >
> > $ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> > ~/gateway-client-trust.jks
> > % Total % Received % Xferd Average Speed Time Time Time
> > Current
> > Dload Upload Total Spent Left
> > Speed
> > 100 674 100 674 0 0 43602 0 --:--:-- --:--:-- --:--:--
> > 44933
> >
> > $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> > Enter username: guest
> > Enter password:
> > [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
> >
> >
> > Thanks,
> > Sandor
> >
>
Re: [DISCUSS] Knox Shell command for downloading the public cert from
a Knox instance
Posted by Kevin Risden <kr...@apache.org>.
I'm not a fan of adding a new endpoint. This port would need to be
configurable. We would need to ensure that it always points to the correct
location.
Instead, can we download the cert from the existing HTTPS endpoint? We
would have to not trust the TLS connection to pull the public cert, but
this is the same as pulling the truststore over the HTTP endpoint you
proposed. This will guarantee we don't have to do anything special on
startup and will work with any configured TLS certificate.
Kevin Risden
On Mon, Feb 25, 2019 at 9:44 AM Sandor Molnar <sm...@cloudera.com.invalid>
wrote:
> Hi folks,
>
> I've just started to think about how to resolve
> https://issues.apache.org/jira/browse/KNOX-1418 and an approach could be:
>
> 1.) Server-side changes
> I'm thinking of starting a new embedded Jetty instance when the gateway
> server starts on a pre-configured port (e.g. 8100) with a simple HTTP
> connector. This connector would have only one handler: Jetty's
> ResourceHandler (
>
> https://www.eclipse.org/jetty/javadoc/9.4.8.v20171121/index.html?org/eclipse/jetty/server/handler/ResourceHandler.html
> ).
> Whenever the gateway is started we would export the public cert of the
> gateway into a dedicated folder (e.g.
> $GATEWAY_HOME/data/security/clientCert/gateway-client-trust.jks). We have
> to configure the ResourceHandler to allow access to this folder only (thus
> nothing else will be exposed through this new endpoint).
>
> 2.) KnoxShell-side changes
> Within KnoxShell we should add a new command that simply hits the new
> endpoint and save the output in the root of the current user (for instance
> curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> ~/gateway-client-trust.jks)
>
> 3.) Optionally, we may change KnoxCLI to inform the end-user about the new
> location of the JKS certificate in case the user executes `knoxcli.sh
> export-cert --type JKS` (it does not make sense to do the same what we
> already have)
>
> Any comments are highly appreciated!
>
> I've already coded a POC and it works as expected:
>
> $ rm -f ~/gateway-client-trust.jks
>
> $ curl http://c7401.ambari.apache.org:8100/gateway-client-trust.jks >
> ~/gateway-client-trust.jks
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 674 100 674 0 0 43602 0 --:--:-- --:--:-- --:--:--
> 44933
>
> $ ./bin/knoxshell.sh samples/ExampleWebHdfsLs.groovy
> Enter username: guest
> Enter password:
> [app-logs, ats, atsv2, hdp, mapred, mr-history, tmp, user]
>
>
> Thanks,
> Sandor
>