You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com> on 2015/06/01 08:32:11 UTC
RE: Unable to start the gateway
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Hi Larry,
First of all a huge thanks for your patience ☺. You are awesome!!
As per your email I have gone through the documentation and found out this format
https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/{service-name}<https://%7bgateway-host%7d:%7bgateway-port%7d/%7bgateway-path%7d/%7bcluster-name%7d/%7bservice-name%7d>
I did the following changes to my config files. I am sharing the files as an attachment. (sample.xml is my new cluster name under topologies directory)
Now I recreated the URL curl -v -k https://localhost:8444/hadoop/sample/WEBHDFS/v1?op=gethomedirectory&user.name=map (this time hadoop is my context path)
And, it gives me the same page not found. Am I doing something wrong with the configs. Let me know if I am missing out something here
* About to connect() to localhost port 8444 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8444 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Jun 02 04:48:46 2015 GMT
* expire date: Jun 01 04:48:46 2016 GMT
* common name: localhost
* issuer: CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
> GET /hadoop/sample/WEBHDFS/v1?op=gethomedirectory&user.name=map HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:8444
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Type: text/html;charset=ISO-8859-1
< Content-Length: 1291
< Server: Jetty(8.1.14.v20131031)
<
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /hadoop/sample/WEBHDFS/v1. Reason:
<pre> Not Found</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
* Connection #0 to host localhost left intact
* Closing connection #0
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Thursday, June 04, 2015 5:52 PM
To: user@knox.apache.org
Subject: Re: Unable to start the gateway
Hi Phani -
The curl error message spells it out pretty well.
Out of the box, the gateway is using a self-signed certificate for SSL.
You either have to add it to an appropriate trust store or turn off SSL validation on your curl command with a "-k".
See the examples in the users guide.
The 404 that you are seeing is related to the URL that you are using.
Unless you added a new topology called single-node, you should be using sandbox.
That application context path is related to the topology file name that you are using to define the cluster that you are accessing.
This should also be spelled out in the users guide pretty well.
Hope that helps.
--larry
On Thu, Jun 4, 2015 at 2:06 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Adding to that here is the configuration I did
sandbox.xml
<service>
<role>WEBHDFS</role>
<url>http://localhost:14000/webhdfs</url>
</service>
gateway-site.xml
<property>
<name>gateway.port</name>
<value>8444</value>
<description>The HTTP port for the Gateway.</description>
</property>
<property>
<name>gateway.path</name>
<value>gateway</value>
<description>The default context path for the gateway.</description>
</property>
When I use curl I get the bellow certificate message. If I am trying to access the link via browser if am seeing the below
[cid:image001.png@01D09EFC.A40A4CB0]
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Thursday, June 04, 2015 11:23 AM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: RE: Unable to start the gateway
Hi Larry,
The gateway seems to be working now. Can you please help me with the below message.
[mapr@cldb-node-01 ~]$ curl -v "https://localhost:8444/gateway/single-node/WEBHDFS/v1?op=gethomedirectory&user.name=mapr"
* About to connect() to localhost port 8444 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8444 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* Issuer certificate is invalid: 'CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US'
* NSS error -8156
* Closing connection #0
* Peer certificate cannot be authenticated with known CA certificates
curl: (60) Peer certificate cannot be authenticated with known CA certificates
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 9:17 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: RE: Unable to start the gateway
Thank you,Larry. I will try these.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com]
Sent: Tuesday, June 02, 2015 8:05 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
I suggest that you take a look at:
* the {GATEWAY_HOME}/samples directory
* WebHDFS with ClientDSL section in http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
* Tests for WebHDFS
The samples directory contains groovy script examples for doing the sorts of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy in particular.
On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I have a simple java api based program which tried to read a file(HDFS). When I run the jar file it can either go via the knox gateway or I should be able to include the gateway information within my java code so that when the program run it would be taken care. I wanted to know if there are any such API’s or examples for the use case.
Also, let me know if there is any other way I can handle this.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 02, 2015 5:00 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master secret, more specific certs, etc
2. Topology related administration is available via REST API that you can read about at: http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
Re: Unable to start the gateway
Posted by larry mccay <lm...@apache.org>.
Hi Phani -
The curl error message spells it out pretty well.
Out of the box, the gateway is using a self-signed certificate for SSL.
You either have to add it to an appropriate trust store or turn off SSL
validation on your curl command with a "-k".
See the examples in the users guide.
The 404 that you are seeing is related to the URL that you are using.
Unless you added a new topology called single-node, you should be using
sandbox.
That application context path is related to the topology file name that you
are using to define the cluster that you are accessing.
This should also be spelled out in the users guide pretty well.
Hope that helps.
--larry
On Thu, Jun 4, 2015 at 2:06 AM, Phani Yadavilli -X (pyadavil) <
pyadavil@cisco.com> wrote:
> Adding to that here is the configuration I did
>
>
>
> *sandbox.xml*
>
>
>
>
>
> <service>
>
> <role>WEBHDFS</role>
>
> <url>http://localhost:14000/webhdfs</url>
>
> </service>
>
>
>
> *gateway-site.xml*
>
>
>
> <property>
>
> <name>gateway.port</name>
>
> <value>8444</value>
>
> <description>The HTTP port for the Gateway.</description>
>
> </property>
>
>
>
> <property>
>
> <name>gateway.path</name>
>
> <value>gateway</value>
>
> <description>The default context path for the
> gateway.</description>
>
> </property>
>
>
>
> When I use curl I get the bellow certificate message. If I am trying to
> access the link via browser if am seeing the below
>
>
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Thursday, June 04, 2015 11:23 AM
>
> *To:* user@knox.apache.org
> *Subject:* RE: Unable to start the gateway
>
>
>
> Hi Larry,
>
>
>
> The gateway seems to be working now. Can you please help me with the below
> message.
>
>
>
> [mapr@cldb-node-01 ~]$ curl -v "
> https://localhost:8444/gateway/single-node/WEBHDFS/v1?op=gethomedirectory&user.name=mapr
> "
>
>
>
> * About to connect() to localhost port 8444 (#0)
>
> * Trying 127.0.0.1... connected
>
> * Connected to localhost (127.0.0.1) port 8444 (#0)
>
> * Initializing NSS with certpath: sql:/etc/pki/nssdb
>
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
>
> CApath: none
>
> * Issuer certificate is invalid:
> 'CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US'
>
> * NSS error -8156
>
> * Closing connection #0
>
> * Peer certificate cannot be authenticated with known CA certificates
>
> curl: (60) Peer certificate cannot be authenticated with known CA
> certificates
>
> More details here: http://curl.haxx.se/docs/sslcerts.html
>
>
>
> curl performs SSL certificate verification by default, using a "bundle"
>
> of Certificate Authority (CA) public keys (CA certs). If the default
>
> bundle file isn't adequate, you can specify an alternate file
>
> using the --cacert option.
>
> If this HTTPS server uses a certificate signed by a CA represented in
>
> the bundle, the certificate verification probably failed due to a
>
> problem with the certificate (it might be expired, or the name might
>
> not match the domain name in the URL).
>
> If you'd like to turn off curl's verification of the certificate, use
>
> the -k (or --insecure) option.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Tuesday, June 02, 2015 9:17 PM
> *To:* user@knox.apache.org
> *Subject:* RE: Unable to start the gateway
>
>
>
> Thank you,Larry. I will try these.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:larry.mccay@gmail.com <la...@gmail.com>]
>
> *Sent:* Tuesday, June 02, 2015 8:05 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> I suggest that you take a look at:
>
> * the {GATEWAY_HOME}/samples directory
>
> * WebHDFS with ClientDSL section in
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
>
> * Tests for WebHDFS
>
>
>
> The samples directory contains groovy script examples for doing the sorts
> of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy
> in particular.
>
>
>
> On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Larry,
>
>
>
> I have a simple java api based program which tried to read a file(HDFS).
> When I run the jar file it can either go via the knox gateway or I should
> be able to include the gateway information within my java code so that when
> the program run it would be taken care. I wanted to know if there are any
> such API’s or examples for the use case.
>
>
>
> Also, let me know if there is any other way I can handle this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:larry.mccay@gmail.com]
> *Sent:* Tuesday, June 02, 2015 5:00 PM
>
>
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Glad to hear that you are up and running now.
>
>
>
> Depending on what sort of configuration you are talking about, you can do
> two separate things:
>
>
>
> 1. You can shell out to the knoxcli.sh script for provisioning the master
> secret, more specific certs, etc
>
> 2. Topology related administration is available via REST API that you can
> read about at:
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
>
>
>
> Hope that is useful.
>
>
>
> --larry
>
>
>
> On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Btw, do we have any JAVA API to configure knox from the code level.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Tuesday, June 02, 2015 2:47 PM
>
>
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Hi Larry,
>
>
>
> The issue is resolved.
>
>
>
> Thanks for the help.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 6:26 PM
> *To:* Phani Yadavilli -X (pyadavil)
> *Cc:* vinayshukla@gmail.com; user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> That usually means that it is already running - or you have something else
> running on port 33389.
>
>
>
> Try "ps -ef|grep ldap" and you should see that it is already running.
>
>
>
> On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Larry,
>
>
>
> I tried the below steps. This time the gateway started but the ladp is not
> starting. Please find the error logs as below
>
>
>
> [knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
>
> Starting LDAP failed.
>
>
>
> [knox@cldb-node-01 logs]$ cat ldap.err
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Monday, June 01, 2015 5:12 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Thanks larry. I will try this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 5:10 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Phani -
>
>
>
> I have seen issues like this when the keystore was created with one JDK -
> like IBM - and the runtime tries to start with openjdk or oracle JDK.
>
> Check that those versions are the same.
>
>
>
> If you haven't provisioned your own certificate then you may want to try
> removing the keystores in {GATEWAY_HOME}/data/security/keystores and
> allowing them to be recreated.
>
>
>
> If the master secret that you provided doesn't match the one used to
> create them when you first started it then this would happen as well.
>
> Recreating them with the new master secret will fix that - if it is the
> case.
>
>
>
> HTH.
>
>
>
> --larry
>
>
>
>
>
> On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Thank you larry for the info.
>
>
>
> Vinay,
>
>
>
> I am still not able to start the gateway. I used the below process to
> change the password. Am I still missing something here.
>
>
>
> [knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
>
>
> ***************************************************************************************************
>
> You have indicated that you would like to persist the master secret for
> this service instance.
>
> Be aware that this is less secure than manually entering the secret on
> startup.
>
> The persisted file will be encrypted and primarily protected through OS
> permissions.
>
>
> ***************************************************************************************************
>
> Enter master secret:
>
> Enter master secret again:
>
> Master secret has been persisted to disk.
>
> [knox@cldb-node-01 bin]$ ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org]
> *Sent:* Friday, May 29, 2015 10:27 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani -
>
>
>
> You should find that Knox WEBHDFS support is compatible with any
> implementation of the HDFS REST API.
>
> Things like HttpFS should work in place of WebHDFS if your distro is
> missing explicit support for it.
>
>
>
> Ranger service level authorization would still work with Knox in this case
> but may not have lower level authorization enforcement that requires NN -
> again depending on your distro's support.
>
>
>
> Hope this is helpful to you.
>
>
>
> --larry
>
>
>
> On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Vinay,
>
>
>
> Thank you. I will try that. I have another question.
>
> We are using the MapR cluster where we do not have NameNode. Instead we
> have CLDB. Is there any workaround I can do to make the ranger and knox
> work with the cluster.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Vinay Shukla [mailto:vinayshukla@gmail.com]
> *Sent:* Friday, May 29, 2015 8:29 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> You missed the below step needed once before starting gateway for the
> first time.
>
>
>
> bin/knoxcli.sh create-master
>
>
>
> Thx,
>
> Vinay
>
>
>
> On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Kevin,
>
>
>
> Please find the log attached.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Kevin Minder [mailto:kevin.minder@hortonworks.com]
> *Sent:* Friday, May 29, 2015 6:30 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani,
>
> Could you take a look at the log file (logs/gateway.log) and show us the
> content of that. With that we should be able to help.
>
> Kevin.
>
>
>
> *From: *"Phani Yadavilli -X (pyadavil)" <py...@cisco.com>
> *Reply-To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Date: *Friday, May 29, 2015 at 2:54 AM
> *To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Subject: *Unable to start the gateway
>
>
>
> Hi Team,
>
>
>
> I am a newbie to the knox. I am trying to setup knox in our environment as
> a part of POC. When I am trying to run the
>
>
>
> ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> I am unable to start the gateway. Could you please help me understand and
> debug the issue.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
>
>
>
>
>
>
>
>
>
>
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Adding to that here is the configuration I did
sandbox.xml
<service>
<role>WEBHDFS</role>
<url>http://localhost:14000/webhdfs</url>
</service>
gateway-site.xml
<property>
<name>gateway.port</name>
<value>8444</value>
<description>The HTTP port for the Gateway.</description>
</property>
<property>
<name>gateway.path</name>
<value>gateway</value>
<description>The default context path for the gateway.</description>
</property>
When I use curl I get the bellow certificate message. If I am trying to access the link via browser if am seeing the below
[cid:image001.png@01D09EBA.B7A08380]
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Thursday, June 04, 2015 11:23 AM
To: user@knox.apache.org
Subject: RE: Unable to start the gateway
Hi Larry,
The gateway seems to be working now. Can you please help me with the below message.
[mapr@cldb-node-01 ~]$ curl -v "https://localhost:8444/gateway/single-node/WEBHDFS/v1?op=gethomedirectory&user.name=mapr"
* About to connect() to localhost port 8444 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8444 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* Issuer certificate is invalid: 'CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US'
* NSS error -8156
* Closing connection #0
* Peer certificate cannot be authenticated with known CA certificates
curl: (60) Peer certificate cannot be authenticated with known CA certificates
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 9:17 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: RE: Unable to start the gateway
Thank you,Larry. I will try these.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com]
Sent: Tuesday, June 02, 2015 8:05 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
I suggest that you take a look at:
* the {GATEWAY_HOME}/samples directory
* WebHDFS with ClientDSL section in http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
* Tests for WebHDFS
The samples directory contains groovy script examples for doing the sorts of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy in particular.
On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I have a simple java api based program which tried to read a file(HDFS). When I run the jar file it can either go via the knox gateway or I should be able to include the gateway information within my java code so that when the program run it would be taken care. I wanted to know if there are any such API’s or examples for the use case.
Also, let me know if there is any other way I can handle this.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 02, 2015 5:00 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master secret, more specific certs, etc
2. Topology related administration is available via REST API that you can read about at: http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Hi Larry,
The gateway seems to be working now. Can you please help me with the below message.
[mapr@cldb-node-01 ~]$ curl -v "https://localhost:8444/gateway/single-node/WEBHDFS/v1?op=gethomedirectory&user.name=mapr"
* About to connect() to localhost port 8444 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8444 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* Issuer certificate is invalid: 'CN=localhost,OU=Test,O=Hadoop,L=Test,ST=Test,C=US'
* NSS error -8156
* Closing connection #0
* Peer certificate cannot be authenticated with known CA certificates
curl: (60) Peer certificate cannot be authenticated with known CA certificates
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 9:17 PM
To: user@knox.apache.org
Subject: RE: Unable to start the gateway
Thank you,Larry. I will try these.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com]
Sent: Tuesday, June 02, 2015 8:05 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
I suggest that you take a look at:
* the {GATEWAY_HOME}/samples directory
* WebHDFS with ClientDSL section in http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
* Tests for WebHDFS
The samples directory contains groovy script examples for doing the sorts of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy in particular.
On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I have a simple java api based program which tried to read a file(HDFS). When I run the jar file it can either go via the knox gateway or I should be able to include the gateway information within my java code so that when the program run it would be taken care. I wanted to know if there are any such API’s or examples for the use case.
Also, let me know if there is any other way I can handle this.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 02, 2015 5:00 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master secret, more specific certs, etc
2. Topology related administration is available via REST API that you can read about at: http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Thank you,Larry. I will try these.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com]
Sent: Tuesday, June 02, 2015 8:05 PM
To: user@knox.apache.org
Subject: Re: Unable to start the gateway
I suggest that you take a look at:
* the {GATEWAY_HOME}/samples directory
* WebHDFS with ClientDSL section in http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
* Tests for WebHDFS
The samples directory contains groovy script examples for doing the sorts of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy in particular.
On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I have a simple java api based program which tried to read a file(HDFS). When I run the jar file it can either go via the knox gateway or I should be able to include the gateway information within my java code so that when the program run it would be taken care. I wanted to know if there are any such API’s or examples for the use case.
Also, let me know if there is any other way I can handle this.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com<ma...@gmail.com>]
Sent: Tuesday, June 02, 2015 5:00 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master secret, more specific certs, etc
2. Topology related administration is available via REST API that you can read about at: http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
Re: Unable to start the gateway
Posted by larry mccay <la...@gmail.com>.
I suggest that you take a look at:
* the {GATEWAY_HOME}/samples directory
* WebHDFS with ClientDSL section in
http://knox.apache.org/books/knox-0-6-0/user-guide.html#WebHDFS
* Tests for WebHDFS
The samples directory contains groovy script examples for doing the sorts
of things that you are talking about - look at: ExampleWebHdfsPutGet.groovy
in particular.
On Tue, Jun 2, 2015 at 10:23 AM, Phani Yadavilli -X (pyadavil) <
pyadavil@cisco.com> wrote:
> Hi Larry,
>
>
>
> I have a simple java api based program which tried to read a file(HDFS).
> When I run the jar file it can either go via the knox gateway or I should
> be able to include the gateway information within my java code so that when
> the program run it would be taken care. I wanted to know if there are any
> such API’s or examples for the use case.
>
>
>
> Also, let me know if there is any other way I can handle this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:larry.mccay@gmail.com]
> *Sent:* Tuesday, June 02, 2015 5:00 PM
>
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Glad to hear that you are up and running now.
>
>
>
> Depending on what sort of configuration you are talking about, you can do
> two separate things:
>
>
>
> 1. You can shell out to the knoxcli.sh script for provisioning the master
> secret, more specific certs, etc
>
> 2. Topology related administration is available via REST API that you can
> read about at:
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
>
>
>
> Hope that is useful.
>
>
>
> --larry
>
>
>
> On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Btw, do we have any JAVA API to configure knox from the code level.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Tuesday, June 02, 2015 2:47 PM
>
>
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Hi Larry,
>
>
>
> The issue is resolved.
>
>
>
> Thanks for the help.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 6:26 PM
> *To:* Phani Yadavilli -X (pyadavil)
> *Cc:* vinayshukla@gmail.com; user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> That usually means that it is already running - or you have something else
> running on port 33389.
>
>
>
> Try "ps -ef|grep ldap" and you should see that it is already running.
>
>
>
> On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Larry,
>
>
>
> I tried the below steps. This time the gateway started but the ladp is not
> starting. Please find the error logs as below
>
>
>
> [knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
>
> Starting LDAP failed.
>
>
>
> [knox@cldb-node-01 logs]$ cat ldap.err
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Monday, June 01, 2015 5:12 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Thanks larry. I will try this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 5:10 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Phani -
>
>
>
> I have seen issues like this when the keystore was created with one JDK -
> like IBM - and the runtime tries to start with openjdk or oracle JDK.
>
> Check that those versions are the same.
>
>
>
> If you haven't provisioned your own certificate then you may want to try
> removing the keystores in {GATEWAY_HOME}/data/security/keystores and
> allowing them to be recreated.
>
>
>
> If the master secret that you provided doesn't match the one used to
> create them when you first started it then this would happen as well.
>
> Recreating them with the new master secret will fix that - if it is the
> case.
>
>
>
> HTH.
>
>
>
> --larry
>
>
>
>
>
> On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Thank you larry for the info.
>
>
>
> Vinay,
>
>
>
> I am still not able to start the gateway. I used the below process to
> change the password. Am I still missing something here.
>
>
>
> [knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
>
>
> ***************************************************************************************************
>
> You have indicated that you would like to persist the master secret for
> this service instance.
>
> Be aware that this is less secure than manually entering the secret on
> startup.
>
> The persisted file will be encrypted and primarily protected through OS
> permissions.
>
>
> ***************************************************************************************************
>
> Enter master secret:
>
> Enter master secret again:
>
> Master secret has been persisted to disk.
>
> [knox@cldb-node-01 bin]$ ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org]
> *Sent:* Friday, May 29, 2015 10:27 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani -
>
>
>
> You should find that Knox WEBHDFS support is compatible with any
> implementation of the HDFS REST API.
>
> Things like HttpFS should work in place of WebHDFS if your distro is
> missing explicit support for it.
>
>
>
> Ranger service level authorization would still work with Knox in this case
> but may not have lower level authorization enforcement that requires NN -
> again depending on your distro's support.
>
>
>
> Hope this is helpful to you.
>
>
>
> --larry
>
>
>
> On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Vinay,
>
>
>
> Thank you. I will try that. I have another question.
>
> We are using the MapR cluster where we do not have NameNode. Instead we
> have CLDB. Is there any workaround I can do to make the ranger and knox
> work with the cluster.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Vinay Shukla [mailto:vinayshukla@gmail.com]
> *Sent:* Friday, May 29, 2015 8:29 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> You missed the below step needed once before starting gateway for the
> first time.
>
>
>
> bin/knoxcli.sh create-master
>
>
>
> Thx,
>
> Vinay
>
>
>
> On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Kevin,
>
>
>
> Please find the log attached.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Kevin Minder [mailto:kevin.minder@hortonworks.com]
> *Sent:* Friday, May 29, 2015 6:30 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani,
>
> Could you take a look at the log file (logs/gateway.log) and show us the
> content of that. With that we should be able to help.
>
> Kevin.
>
>
>
> *From: *"Phani Yadavilli -X (pyadavil)" <py...@cisco.com>
> *Reply-To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Date: *Friday, May 29, 2015 at 2:54 AM
> *To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Subject: *Unable to start the gateway
>
>
>
> Hi Team,
>
>
>
> I am a newbie to the knox. I am trying to setup knox in our environment as
> a part of POC. When I am trying to run the
>
>
>
> ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> I am unable to start the gateway. Could you please help me understand and
> debug the issue.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
>
>
>
>
>
>
>
>
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Hi Larry,
I have a simple java api based program which tried to read a file(HDFS). When I run the jar file it can either go via the knox gateway or I should be able to include the gateway information within my java code so that when the program run it would be taken care. I wanted to know if there are any such API’s or examples for the use case.
Also, let me know if there is any other way I can handle this.
Regards
Phani Kumar
From: larry mccay [mailto:larry.mccay@gmail.com]
Sent: Tuesday, June 02, 2015 5:00 PM
To: user@knox.apache.org
Cc: vinayshukla@gmail.com
Subject: Re: Unable to start the gateway
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master secret, more specific certs, etc
2. Topology related administration is available via REST API that you can read about at: http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
Re: Unable to start the gateway
Posted by larry mccay <la...@gmail.com>.
Glad to hear that you are up and running now.
Depending on what sort of configuration you are talking about, you can do
two separate things:
1. You can shell out to the knoxcli.sh script for provisioning the master
secret, more specific certs, etc
2. Topology related administration is available via REST API that you can
read about at:
http://knox.apache.org/books/knox-0-6-0/user-guide.html#Admin+API
Hope that is useful.
--larry
On Tue, Jun 2, 2015 at 5:19 AM, Phani Yadavilli -X (pyadavil) <
pyadavil@cisco.com> wrote:
> Btw, do we have any JAVA API to configure knox from the code level.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Tuesday, June 02, 2015 2:47 PM
>
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Hi Larry,
>
>
>
> The issue is resolved.
>
>
>
> Thanks for the help.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 6:26 PM
> *To:* Phani Yadavilli -X (pyadavil)
> *Cc:* vinayshukla@gmail.com; user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> That usually means that it is already running - or you have something else
> running on port 33389.
>
>
>
> Try "ps -ef|grep ldap" and you should see that it is already running.
>
>
>
> On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Larry,
>
>
>
> I tried the below steps. This time the gateway started but the ladp is not
> starting. Please find the error logs as below
>
>
>
> [knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
>
> Starting LDAP failed.
>
>
>
> [knox@cldb-node-01 logs]$ cat ldap.err
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Monday, June 01, 2015 5:12 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Thanks larry. I will try this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 5:10 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Phani -
>
>
>
> I have seen issues like this when the keystore was created with one JDK -
> like IBM - and the runtime tries to start with openjdk or oracle JDK.
>
> Check that those versions are the same.
>
>
>
> If you haven't provisioned your own certificate then you may want to try
> removing the keystores in {GATEWAY_HOME}/data/security/keystores and
> allowing them to be recreated.
>
>
>
> If the master secret that you provided doesn't match the one used to
> create them when you first started it then this would happen as well.
>
> Recreating them with the new master secret will fix that - if it is the
> case.
>
>
>
> HTH.
>
>
>
> --larry
>
>
>
>
>
> On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Thank you larry for the info.
>
>
>
> Vinay,
>
>
>
> I am still not able to start the gateway. I used the below process to
> change the password. Am I still missing something here.
>
>
>
> [knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
>
>
> ***************************************************************************************************
>
> You have indicated that you would like to persist the master secret for
> this service instance.
>
> Be aware that this is less secure than manually entering the secret on
> startup.
>
> The persisted file will be encrypted and primarily protected through OS
> permissions.
>
>
> ***************************************************************************************************
>
> Enter master secret:
>
> Enter master secret again:
>
> Master secret has been persisted to disk.
>
> [knox@cldb-node-01 bin]$ ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org]
> *Sent:* Friday, May 29, 2015 10:27 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani -
>
>
>
> You should find that Knox WEBHDFS support is compatible with any
> implementation of the HDFS REST API.
>
> Things like HttpFS should work in place of WebHDFS if your distro is
> missing explicit support for it.
>
>
>
> Ranger service level authorization would still work with Knox in this case
> but may not have lower level authorization enforcement that requires NN -
> again depending on your distro's support.
>
>
>
> Hope this is helpful to you.
>
>
>
> --larry
>
>
>
> On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Vinay,
>
>
>
> Thank you. I will try that. I have another question.
>
> We are using the MapR cluster where we do not have NameNode. Instead we
> have CLDB. Is there any workaround I can do to make the ranger and knox
> work with the cluster.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Vinay Shukla [mailto:vinayshukla@gmail.com]
> *Sent:* Friday, May 29, 2015 8:29 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> You missed the below step needed once before starting gateway for the
> first time.
>
>
>
> bin/knoxcli.sh create-master
>
>
>
> Thx,
>
> Vinay
>
>
>
> On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Kevin,
>
>
>
> Please find the log attached.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Kevin Minder [mailto:kevin.minder@hortonworks.com]
> *Sent:* Friday, May 29, 2015 6:30 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani,
>
> Could you take a look at the log file (logs/gateway.log) and show us the
> content of that. With that we should be able to help.
>
> Kevin.
>
>
>
> *From: *"Phani Yadavilli -X (pyadavil)" <py...@cisco.com>
> *Reply-To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Date: *Friday, May 29, 2015 at 2:54 AM
> *To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Subject: *Unable to start the gateway
>
>
>
> Hi Team,
>
>
>
> I am a newbie to the knox. I am trying to setup knox in our environment as
> a part of POC. When I am trying to run the
>
>
>
> ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> I am unable to start the gateway. Could you please help me understand and
> debug the issue.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
>
>
>
>
>
>
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Btw, do we have any JAVA API to configure knox from the code level.
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Tuesday, June 02, 2015 2:47 PM
To: user@knox.apache.org
Cc: vinayshukla@gmail.com
Subject: RE: Unable to start the gateway
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com<ma...@gmail.com>; user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Hi Larry,
The issue is resolved.
Thanks for the help.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 6:26 PM
To: Phani Yadavilli -X (pyadavil)
Cc: vinayshukla@gmail.com; user@knox.apache.org
Subject: Re: Unable to start the gateway
That usually means that it is already running - or you have something else running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
Re: Unable to start the gateway
Posted by larry mccay <lm...@apache.org>.
That usually means that it is already running - or you have something else
running on port 33389.
Try "ps -ef|grep ldap" and you should see that it is already running.
On Mon, Jun 1, 2015 at 7:52 AM, Phani Yadavilli -X (pyadavil) <
pyadavil@cisco.com> wrote:
> Hi Larry,
>
>
>
> I tried the below steps. This time the gateway started but the ladp is not
> starting. Please find the error logs as below
>
>
>
> [knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
>
> Starting LDAP failed.
>
>
>
> [knox@cldb-node-01 logs]$ cat ldap.err
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
> Exception in thread "main" java.lang.reflect.InvocationTargetException
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
>
> at
> org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
>
> at
> org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
>
> Caused by: java.net.BindException: Address already in use
>
> at java.net.PlainSocketImpl.socketBind(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
>
> at java.net.ServerSocket.bind(ServerSocket.java:376)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:237)
>
> at java.net.ServerSocket.<init>(ServerSocket.java:128)
>
> at
> org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
>
> ... 9 more
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Phani Yadavilli -X (pyadavil)
> *Sent:* Monday, June 01, 2015 5:12 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* RE: Unable to start the gateway
>
>
>
> Thanks larry. I will try this.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org <lm...@apache.org>]
> *Sent:* Monday, June 01, 2015 5:10 PM
> *To:* user@knox.apache.org
> *Cc:* vinayshukla@gmail.com
> *Subject:* Re: Unable to start the gateway
>
>
>
> Phani -
>
>
>
> I have seen issues like this when the keystore was created with one JDK -
> like IBM - and the runtime tries to start with openjdk or oracle JDK.
>
> Check that those versions are the same.
>
>
>
> If you haven't provisioned your own certificate then you may want to try
> removing the keystores in {GATEWAY_HOME}/data/security/keystores and
> allowing them to be recreated.
>
>
>
> If the master secret that you provided doesn't match the one used to
> create them when you first started it then this would happen as well.
>
> Recreating them with the new master secret will fix that - if it is the
> case.
>
>
>
> HTH.
>
>
>
> --larry
>
>
>
>
>
> On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Thank you larry for the info.
>
>
>
> Vinay,
>
>
>
> I am still not able to start the gateway. I used the below process to
> change the password. Am I still missing something here.
>
>
>
> [knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
>
>
> ***************************************************************************************************
>
> You have indicated that you would like to persist the master secret for
> this service instance.
>
> Be aware that this is less secure than manually entering the secret on
> startup.
>
> The persisted file will be encrypted and primarily protected through OS
> permissions.
>
>
> ***************************************************************************************************
>
> Enter master secret:
>
> Enter master secret again:
>
> Master secret has been persisted to disk.
>
> [knox@cldb-node-01 bin]$ ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org]
> *Sent:* Friday, May 29, 2015 10:27 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani -
>
>
>
> You should find that Knox WEBHDFS support is compatible with any
> implementation of the HDFS REST API.
>
> Things like HttpFS should work in place of WebHDFS if your distro is
> missing explicit support for it.
>
>
>
> Ranger service level authorization would still work with Knox in this case
> but may not have lower level authorization enforcement that requires NN -
> again depending on your distro's support.
>
>
>
> Hope this is helpful to you.
>
>
>
> --larry
>
>
>
> On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Vinay,
>
>
>
> Thank you. I will try that. I have another question.
>
> We are using the MapR cluster where we do not have NameNode. Instead we
> have CLDB. Is there any workaround I can do to make the ranger and knox
> work with the cluster.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Vinay Shukla [mailto:vinayshukla@gmail.com]
> *Sent:* Friday, May 29, 2015 8:29 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> You missed the below step needed once before starting gateway for the
> first time.
>
>
>
> bin/knoxcli.sh create-master
>
>
>
> Thx,
>
> Vinay
>
>
>
> On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Kevin,
>
>
>
> Please find the log attached.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Kevin Minder [mailto:kevin.minder@hortonworks.com]
> *Sent:* Friday, May 29, 2015 6:30 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani,
>
> Could you take a look at the log file (logs/gateway.log) and show us the
> content of that. With that we should be able to help.
>
> Kevin.
>
>
>
> *From: *"Phani Yadavilli -X (pyadavil)" <py...@cisco.com>
> *Reply-To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Date: *Friday, May 29, 2015 at 2:54 AM
> *To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Subject: *Unable to start the gateway
>
>
>
> Hi Team,
>
>
>
> I am a newbie to the knox. I am trying to setup knox in our environment as
> a part of POC. When I am trying to run the
>
>
>
> ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> I am unable to start the gateway. Could you please help me understand and
> debug the issue.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
>
>
>
>
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Hi Larry,
I tried the below steps. This time the gateway started but the ladp is not starting. Please find the error logs as below
[knox@cldb-node-01 knox-0.6.0]$ bin/ldap.sh start
Starting LDAP failed.
[knox@cldb-node-01 logs]$ cat ldap.err
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.apache.hadoop.gateway.security.ldap.SimpleLdapDirectoryServer.main(SimpleLdapDirectoryServer.java:102)
... 9 more
Regards
Phani Kumar
From: Phani Yadavilli -X (pyadavil)
Sent: Monday, June 01, 2015 5:12 PM
To: user@knox.apache.org
Cc: vinayshukla@gmail.com
Subject: RE: Unable to start the gateway
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Cc: vinayshukla@gmail.com<ma...@gmail.com>
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
RE: Unable to start the gateway
Posted by "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>.
Thanks larry. I will try this.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org]
Sent: Monday, June 01, 2015 5:10 PM
To: user@knox.apache.org
Cc: vinayshukla@gmail.com
Subject: Re: Unable to start the gateway
Phani -
I have seen issues like this when the keystore was created with one JDK - like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try removing the keystores in {GATEWAY_HOME}/data/security/keystores and allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Thank you larry for the info.
Vinay,
I am still not able to start the gateway. I used the below process to change the password. Am I still missing something here.
[knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
***************************************************************************************************
You have indicated that you would like to persist the master secret for this service instance.
Be aware that this is less secure than manually entering the secret on startup.
The persisted file will be encrypted and primarily protected through OS permissions.
***************************************************************************************************
Enter master secret:
Enter master secret again:
Master secret has been persisted to disk.
[knox@cldb-node-01 bin]$ ./gateway.sh start
Starting Gateway failed.
Regards
Phani Kumar
From: larry mccay [mailto:lmccay@apache.org<ma...@apache.org>]
Sent: Friday, May 29, 2015 10:27 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani -
You should find that Knox WEBHDFS support is compatible with any implementation of the HDFS REST API.
Things like HttpFS should work in place of WebHDFS if your distro is missing explicit support for it.
Ranger service level authorization would still work with Knox in this case but may not have lower level authorization enforcement that requires NN - again depending on your distro's support.
Hope this is helpful to you.
--larry
On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Vinay,
Thank you. I will try that. I have another question.
We are using the MapR cluster where we do not have NameNode. Instead we have CLDB. Is there any workaround I can do to make the ranger and knox work with the cluster.
Regards
Phani Kumar
From: Vinay Shukla [mailto:vinayshukla@gmail.com<ma...@gmail.com>]
Sent: Friday, May 29, 2015 8:29 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
You missed the below step needed once before starting gateway for the first time.
bin/knoxcli.sh create-master
Thx,
Vinay
On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <py...@cisco.com>> wrote:
Hi Kevin,
Please find the log attached.
Regards
Phani Kumar
From: Kevin Minder [mailto:kevin.minder@hortonworks.com<ma...@hortonworks.com>]
Sent: Friday, May 29, 2015 6:30 PM
To: user@knox.apache.org<ma...@knox.apache.org>
Subject: Re: Unable to start the gateway
Hi Phani,
Could you take a look at the log file (logs/gateway.log) and show us the content of that. With that we should be able to help.
Kevin.
From: "Phani Yadavilli -X (pyadavil)" <py...@cisco.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Friday, May 29, 2015 at 2:54 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Unable to start the gateway
Hi Team,
I am a newbie to the knox. I am trying to setup knox in our environment as a part of POC. When I am trying to run the
./gateway.sh start
Starting Gateway failed.
I am unable to start the gateway. Could you please help me understand and debug the issue.
Regards
Phani Kumar
Re: Unable to start the gateway
Posted by larry mccay <lm...@apache.org>.
Phani -
I have seen issues like this when the keystore was created with one JDK -
like IBM - and the runtime tries to start with openjdk or oracle JDK.
Check that those versions are the same.
If you haven't provisioned your own certificate then you may want to try
removing the keystores in {GATEWAY_HOME}/data/security/keystores and
allowing them to be recreated.
If the master secret that you provided doesn't match the one used to create
them when you first started it then this would happen as well.
Recreating them with the new master secret will fix that - if it is the
case.
HTH.
--larry
On Mon, Jun 1, 2015 at 2:32 AM, Phani Yadavilli -X (pyadavil) <
pyadavil@cisco.com> wrote:
> Thank you larry for the info.
>
>
>
> Vinay,
>
>
>
> I am still not able to start the gateway. I used the below process to
> change the password. Am I still missing something here.
>
>
>
> [knox@cldb-node-01 bin]$ ./knoxcli.sh create-master --force
>
>
> ***************************************************************************************************
>
> You have indicated that you would like to persist the master secret for
> this service instance.
>
> Be aware that this is less secure than manually entering the secret on
> startup.
>
> The persisted file will be encrypted and primarily protected through OS
> permissions.
>
>
> ***************************************************************************************************
>
> Enter master secret:
>
> Enter master secret again:
>
> Master secret has been persisted to disk.
>
> [knox@cldb-node-01 bin]$ ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* larry mccay [mailto:lmccay@apache.org]
> *Sent:* Friday, May 29, 2015 10:27 PM
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani -
>
>
>
> You should find that Knox WEBHDFS support is compatible with any
> implementation of the HDFS REST API.
>
> Things like HttpFS should work in place of WebHDFS if your distro is
> missing explicit support for it.
>
>
>
> Ranger service level authorization would still work with Knox in this case
> but may not have lower level authorization enforcement that requires NN -
> again depending on your distro's support.
>
>
>
> Hope this is helpful to you.
>
>
>
> --larry
>
>
>
> On Fri, May 29, 2015 at 11:22 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Vinay,
>
>
>
> Thank you. I will try that. I have another question.
>
> We are using the MapR cluster where we do not have NameNode. Instead we
> have CLDB. Is there any workaround I can do to make the ranger and knox
> work with the cluster.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Vinay Shukla [mailto:vinayshukla@gmail.com]
> *Sent:* Friday, May 29, 2015 8:29 PM
>
>
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> You missed the below step needed once before starting gateway for the
> first time.
>
>
>
> bin/knoxcli.sh create-master
>
>
>
> Thx,
>
> Vinay
>
>
>
> On Fri, May 29, 2015 at 7:13 AM, Phani Yadavilli -X (pyadavil) <
> pyadavil@cisco.com> wrote:
>
> Hi Kevin,
>
>
>
> Please find the log attached.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
> *From:* Kevin Minder [mailto:kevin.minder@hortonworks.com]
> *Sent:* Friday, May 29, 2015 6:30 PM
> *To:* user@knox.apache.org
> *Subject:* Re: Unable to start the gateway
>
>
>
> Hi Phani,
>
> Could you take a look at the log file (logs/gateway.log) and show us the
> content of that. With that we should be able to help.
>
> Kevin.
>
>
>
> *From: *"Phani Yadavilli -X (pyadavil)" <py...@cisco.com>
> *Reply-To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Date: *Friday, May 29, 2015 at 2:54 AM
> *To: *"user@knox.apache.org" <us...@knox.apache.org>
> *Subject: *Unable to start the gateway
>
>
>
> Hi Team,
>
>
>
> I am a newbie to the knox. I am trying to setup knox in our environment as
> a part of POC. When I am trying to run the
>
>
>
> ./gateway.sh start
>
> Starting Gateway failed.
>
>
>
> I am unable to start the gateway. Could you please help me understand and
> debug the issue.
>
>
>
> Regards
>
> Phani Kumar
>
>
>
>
>