You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Vaibhav Kulkarni <va...@gmail.com> on 2018/02/06 11:49:59 UTC
Memory leak in Knox gateway process
Hi,
I am using Knox server gateway (version 0.12.0) with Hortonworks
HDP-2.6.2.0 . There seems to be a memory leak with the knox process
because of which it keeps acquiring more and more memory. My test workload
does very frequent webhdfs rest apis from hundreds of connections. When the
workload is executing, I notice that the knox gateway process memory keeps
increasing.
PID VSZ SZ RSS %CPU TIME CMD
24703 44988224 11247056 2195740 78.8 01:37:02 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44988224 11247056 2195740 78.8 01:37:03 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989420 11247355 2335764 84.3 01:44:13 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989576 11247394 2326344 89.5 01:51:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989744 11247436 2345940 94.8 01:58:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989908 11247477 2515100 100 02:05:13 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990072 11247518 2653252 105 02:12:18 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990244 11247561 2777268 110 02:19:26 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878924 113 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878924 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878872 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878872 111 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878792 111 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878792 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878392 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878392 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878348 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878348 105 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878016 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878016 103 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
Has anyone seen this behaviour with knox ?
Following is my Knox topology, I am using LDAP authentication
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.ldapRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.
KnoxLdapRealm</value>
</param>
<param>
<name>main.ldapRealm.userDnTemplate</name>
<value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</
value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://{{knox_host_name}}:33389</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.
authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
<param>
<name>knox.acl</name>
<value>admin;*;*</value>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>KNOX</role>
</service>
</topology>
Thanks,
Vaibhav Kulkarni
Re: Memory leak in Knox gateway process
Posted by Sandeep Moré <mo...@gmail.com>.
I suspect what you are seeing is a cache, perhaps try dropping cache
' echo 3 > /proc/sys/vm/drop_caches'
would help.
If Knox process was leaking it would have OOMed but that does not mean that
it is strictly staying within the -Xms and -Xmx bounds [1] as you rightly
pointed
this could be because of the thread stacks and native handles.
I would suggest keeping Xms and Xms sizes same.
[1]
https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/garbage_collect.html
On Tue, Feb 6, 2018 at 9:27 AM, Vaibhav Kulkarni <va...@gmail.com> wrote:
> Hi Sandeep,
>
> Yes, actually, I did try that.
>
> APP_MEM_OPTS
> = "-XX:ParallelGCThreads=8 -XX:NewSize=512m -XX:MaxNewSize=1024m
> -XX:+UseConcMarkSweepGC -Xloggc:$APP_HOME_DIR/logs/gc.log-`date
> +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
> -XX:+PrintGCDateStamps -Xms1024m -Xmx2048m -Xss512k
> -XX:HeapDumpPath=$APP_HOME_DIR/logs -XX:+HeapDumpOnOutOfMemoryError -XX:
> CMSInitiatingOccupancyFraction=60 -XX:+UseCMSInitiatingOccupancyOnly
> -XX:MaxDirectMemorySize=1024m"
>
>
> But, despite that the memory used by knox process keeps increasing and
> goes well beyond the Java heap size. So, I definitely think this is a
> memory leak. But, not sure what is the best way to figure out where the
> leak is happening. It is possible that the leak might be happening in one
> of the native libraries (pam etc) . Any pointers will help
>
> Thanks,
>
>
>
>
> On Tue, Feb 6, 2018 at 7:34 PM, Sandeep Moré <mo...@gmail.com>
> wrote:
>
>> Hello Vaibhav,
>>
>> Did you try setting memory options for Knox ?
>> i.e. setting APP_MEM_OPTS in gateway.sh.
>>
>> Sometimes, in cases of large memory GC does not run as frequently as it
>> should.
>>
>> Best
>> Sandeep
>>
>> On Tue, Feb 6, 2018 at 6:49 AM, Vaibhav Kulkarni <va...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>>
>>> I am using Knox server gateway (version 0.12.0) with Hortonworks
>>> HDP-2.6.2.0 . There seems to be a memory leak with the knox process
>>> because of which it keeps acquiring more and more memory. My test workload
>>> does very frequent webhdfs rest apis from hundreds of connections. When the
>>> workload is executing, I notice that the knox gateway process memory keeps
>>> increasing.
>>>
>>>
>>>
>>> PID VSZ SZ RSS %CPU TIME CMD
>>>
>>> 24703 44988224 11247056 2195740 78.8 01:37:02
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/ga
>>>
>>> teway.jar
>>>
>>> 24703 44988224 11247056 2195740 78.8 01:37:03
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/ga
>>>
>>> teway.jar
>>>
>>> 24703 44989420 11247355 2335764 84.3 01:44:13
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/ga
>>>
>>> teway.jar
>>>
>>> 24703 44989576 11247394 2326344 89.5 01:51:04
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/ga
>>>
>>> teway.jar
>>>
>>> 24703 44989744 11247436 2345940 94.8 01:58:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/ga
>>>
>>> teway.jar
>>>
>>> 24703 44989908 11247477 2515100 100 02:05:13
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990072 11247518 2653252 105 02:12:18
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990244 11247561 2777268 110 02:19:26
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878924 113 02:23:04
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878924 112 02:23:04
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878872 112 02:23:04
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878872 111 02:23:04
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878792 111 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878792 110 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878748 110 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878748 110 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878660 109 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878660 109 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878580 108 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878580 108 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878500 107 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878500 107 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878392 107 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878392 106 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878348 106 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878348 105 02:23:05
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878268 105 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878268 105 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878124 104 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878124 104 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878016 104 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>> 24703 44990324 11247581 2878016 103 02:23:06
>>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>>> -jar /usr/hdp/current/knox-server/bin/gat
>>>
>>> eway.jar
>>>
>>>
>>>
>>> Has anyone seen this behaviour with knox ?
>>>
>>>
>>> Following is my Knox topology, I am using LDAP authentication
>>>
>>>
>>>
>>> <topology>
>>>
>>>
>>> <gateway>
>>>
>>>
>>> <provider>
>>>
>>> <role>authentication</role>
>>>
>>> <name>ShiroProvider</name>
>>>
>>> <enabled>true</enabled>
>>>
>>> <param>
>>>
>>> <name>sessionTimeout</name>
>>>
>>> <value>30</value>
>>>
>>> </param>
>>>
>>> <param>
>>>
>>> <name>main.ldapRealm</name>
>>>
>>> <value>org.apache.hadoop.gatew
>>> ay.shirorealm.KnoxLdapRealm</value>
>>>
>>> </param>
>>>
>>> <param>
>>>
>>> <name>main.ldapRealm.userDnTemplate</name>
>>>
>>> <value>uid={0},ou=people,dc=ha
>>> doop,dc=apache,dc=org</value>
>>>
>>> </param>
>>>
>>> <param>
>>>
>>> <name>main.ldapRealm.contextFactory.url</name>
>>>
>>> <value>ldap://{{knox_host_name}}:33389</value>
>>>
>>> </param>
>>>
>>> <param>
>>>
>>> <name>main.ldapRealm.contextFa
>>> ctory.authenticationMechanism</name>
>>>
>>> <value>simple</value>
>>>
>>> </param>
>>>
>>> <param>
>>>
>>> <name>urls./**</name>
>>>
>>> <value>authcBasic</value>
>>>
>>> </param>
>>>
>>> </provider>
>>>
>>>
>>> <provider>
>>>
>>> <role>authorization</role>
>>>
>>> <name>AclsAuthz</name>
>>>
>>> <enabled>true</enabled>
>>>
>>> <param>
>>>
>>> <name>knox.acl</name>
>>>
>>> <value>admin;*;*</value>
>>>
>>> </param>
>>>
>>> </provider>
>>>
>>>
>>> <provider>
>>>
>>> <role>identity-assertion</role>
>>>
>>> <name>Default</name>
>>>
>>> <enabled>true</enabled>
>>>
>>> </provider>
>>>
>>>
>>> </gateway>
>>>
>>>
>>> <service>
>>>
>>> <role>KNOX</role>
>>>
>>> </service>
>>>
>>>
>>> </topology>
>>>
>>>
>>> Thanks,
>>>
>>> Vaibhav Kulkarni
>>>
>>>
>>
>
Re: Memory leak in Knox gateway process
Posted by Vaibhav Kulkarni <va...@gmail.com>.
Hi Sandeep,
Yes, actually, I did try that.
APP_MEM_OPTS
= "-XX:ParallelGCThreads=8 -XX:NewSize=512m -XX:MaxNewSize=1024m
-XX:+UseConcMarkSweepGC -Xloggc:$APP_HOME_DIR/logs/gc.log-`date
+'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
-XX:+PrintGCDateStamps -Xms1024m -Xmx2048m -Xss512k
-XX:HeapDumpPath=$APP_HOME_DIR/logs
-XX:+HeapDumpOnOutOfMemoryError -XX:CMSInitiatingOccupancyFraction=60
-XX:+UseCMSInitiatingOccupancyOnly -XX:MaxDirectMemorySize=1024m"
But, despite that the memory used by knox process keeps increasing and goes
well beyond the Java heap size. So, I definitely think this is a memory
leak. But, not sure what is the best way to figure out where the leak is
happening. It is possible that the leak might be happening in one of the
native libraries (pam etc) . Any pointers will help
Thanks,
On Tue, Feb 6, 2018 at 7:34 PM, Sandeep Moré <mo...@gmail.com> wrote:
> Hello Vaibhav,
>
> Did you try setting memory options for Knox ?
> i.e. setting APP_MEM_OPTS in gateway.sh.
>
> Sometimes, in cases of large memory GC does not run as frequently as it
> should.
>
> Best
> Sandeep
>
> On Tue, Feb 6, 2018 at 6:49 AM, Vaibhav Kulkarni <va...@gmail.com> wrote:
>
>> Hi,
>>
>>
>> I am using Knox server gateway (version 0.12.0) with Hortonworks
>> HDP-2.6.2.0 . There seems to be a memory leak with the knox process
>> because of which it keeps acquiring more and more memory. My test workload
>> does very frequent webhdfs rest apis from hundreds of connections. When the
>> workload is executing, I notice that the knox gateway process memory keeps
>> increasing.
>>
>>
>>
>> PID VSZ SZ RSS %CPU TIME CMD
>>
>> 24703 44988224 11247056 2195740 78.8 01:37:02
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/ga
>>
>> teway.jar
>>
>> 24703 44988224 11247056 2195740 78.8 01:37:03
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/ga
>>
>> teway.jar
>>
>> 24703 44989420 11247355 2335764 84.3 01:44:13
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/ga
>>
>> teway.jar
>>
>> 24703 44989576 11247394 2326344 89.5 01:51:04
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/ga
>>
>> teway.jar
>>
>> 24703 44989744 11247436 2345940 94.8 01:58:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/ga
>>
>> teway.jar
>>
>> 24703 44989908 11247477 2515100 100 02:05:13
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990072 11247518 2653252 105 02:12:18
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990244 11247561 2777268 110 02:19:26
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878924 113 02:23:04
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878924 112 02:23:04
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878872 112 02:23:04
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878872 111 02:23:04
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878792 111 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878792 110 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878748 110 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878748 110 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878660 109 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878660 109 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878580 108 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878580 108 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878500 107 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878500 107 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878392 107 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878392 106 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878348 106 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878348 105 02:23:05
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878268 105 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878268 105 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878124 104 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878124 104 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878016 104 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>> 24703 44990324 11247581 2878016 103 02:23:06
>> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
>> -jar /usr/hdp/current/knox-server/bin/gat
>>
>> eway.jar
>>
>>
>>
>> Has anyone seen this behaviour with knox ?
>>
>>
>> Following is my Knox topology, I am using LDAP authentication
>>
>>
>>
>> <topology>
>>
>>
>> <gateway>
>>
>>
>> <provider>
>>
>> <role>authentication</role>
>>
>> <name>ShiroProvider</name>
>>
>> <enabled>true</enabled>
>>
>> <param>
>>
>> <name>sessionTimeout</name>
>>
>> <value>30</value>
>>
>> </param>
>>
>> <param>
>>
>> <name>main.ldapRealm</name>
>>
>> <value>org.apache.hadoop.gatew
>> ay.shirorealm.KnoxLdapRealm</value>
>>
>> </param>
>>
>> <param>
>>
>> <name>main.ldapRealm.userDnTemplate</name>
>>
>> <value>uid={0},ou=people,dc=ha
>> doop,dc=apache,dc=org</value>
>>
>> </param>
>>
>> <param>
>>
>> <name>main.ldapRealm.contextFactory.url</name>
>>
>> <value>ldap://{{knox_host_name}}:33389</value>
>>
>> </param>
>>
>> <param>
>>
>> <name>main.ldapRealm.contextFa
>> ctory.authenticationMechanism</name>
>>
>> <value>simple</value>
>>
>> </param>
>>
>> <param>
>>
>> <name>urls./**</name>
>>
>> <value>authcBasic</value>
>>
>> </param>
>>
>> </provider>
>>
>>
>> <provider>
>>
>> <role>authorization</role>
>>
>> <name>AclsAuthz</name>
>>
>> <enabled>true</enabled>
>>
>> <param>
>>
>> <name>knox.acl</name>
>>
>> <value>admin;*;*</value>
>>
>> </param>
>>
>> </provider>
>>
>>
>> <provider>
>>
>> <role>identity-assertion</role>
>>
>> <name>Default</name>
>>
>> <enabled>true</enabled>
>>
>> </provider>
>>
>>
>> </gateway>
>>
>>
>> <service>
>>
>> <role>KNOX</role>
>>
>> </service>
>>
>>
>> </topology>
>>
>>
>> Thanks,
>>
>> Vaibhav Kulkarni
>>
>>
>
Re: Memory leak in Knox gateway process
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Vaibhav,
Did you try setting memory options for Knox ?
i.e. setting APP_MEM_OPTS in gateway.sh.
Sometimes, in cases of large memory GC does not run as frequently as it
should.
Best
Sandeep
On Tue, Feb 6, 2018 at 6:49 AM, Vaibhav Kulkarni <va...@gmail.com> wrote:
> Hi,
>
>
> I am using Knox server gateway (version 0.12.0) with Hortonworks
> HDP-2.6.2.0 . There seems to be a memory leak with the knox process
> because of which it keeps acquiring more and more memory. My test workload
> does very frequent webhdfs rest apis from hundreds of connections. When the
> workload is executing, I notice that the knox gateway process memory keeps
> increasing.
>
>
>
> PID VSZ SZ RSS %CPU TIME CMD
>
> 24703 44988224 11247056 2195740 78.8 01:37:02
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
> -jar /usr/hdp/current/knox-server/bin/ga
>
> teway.jar
>
> 24703 44988224 11247056 2195740 78.8 01:37:03
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
> -jar /usr/hdp/current/knox-server/bin/ga
>
> teway.jar
>
> 24703 44989420 11247355 2335764 84.3 01:44:13
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
> -jar /usr/hdp/current/knox-server/bin/ga
>
> teway.jar
>
> 24703 44989576 11247394 2326344 89.5 01:51:04
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
> -jar /usr/hdp/current/knox-server/bin/ga
>
> teway.jar
>
> 24703 44989744 11247436 2345940 94.8 01:58:06
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.library.path=/usr/hdp/current/knox-server/ext/native
> -jar /usr/hdp/current/knox-server/bin/ga
>
> teway.jar
>
> 24703 44989908 11247477 2515100 100 02:05:13 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990072 11247518 2653252 105 02:12:18 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990244 11247561 2777268 110 02:19:26 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878924 113 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878924 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878872 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878872 111 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878792 111 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878792 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878392 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878392 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878348 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878348 105 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878016 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
> 24703 44990324 11247581 2878016 103 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
> -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
> /usr/hdp/current/knox-server/bin/gat
>
> eway.jar
>
>
>
> Has anyone seen this behaviour with knox ?
>
>
> Following is my Knox topology, I am using LDAP authentication
>
>
>
> <topology>
>
>
> <gateway>
>
>
> <provider>
>
> <role>authentication</role>
>
> <name>ShiroProvider</name>
>
> <enabled>true</enabled>
>
> <param>
>
> <name>sessionTimeout</name>
>
> <value>30</value>
>
> </param>
>
> <param>
>
> <name>main.ldapRealm</name>
>
> <value>org.apache.hadoop.gatew
> ay.shirorealm.KnoxLdapRealm</value>
>
> </param>
>
> <param>
>
> <name>main.ldapRealm.userDnTemplate</name>
>
> <value>uid={0},ou=people,dc=ha
> doop,dc=apache,dc=org</value>
>
> </param>
>
> <param>
>
> <name>main.ldapRealm.contextFactory.url</name>
>
> <value>ldap://{{knox_host_name}}:33389</value>
>
> </param>
>
> <param>
>
> <name>main.ldapRealm.contextFa
> ctory.authenticationMechanism</name>
>
> <value>simple</value>
>
> </param>
>
> <param>
>
> <name>urls./**</name>
>
> <value>authcBasic</value>
>
> </param>
>
> </provider>
>
>
> <provider>
>
> <role>authorization</role>
>
> <name>AclsAuthz</name>
>
> <enabled>true</enabled>
>
> <param>
>
> <name>knox.acl</name>
>
> <value>admin;*;*</value>
>
> </param>
>
> </provider>
>
>
> <provider>
>
> <role>identity-assertion</role>
>
> <name>Default</name>
>
> <enabled>true</enabled>
>
> </provider>
>
>
> </gateway>
>
>
> <service>
>
> <role>KNOX</role>
>
> </service>
>
>
> </topology>
>
>
> Thanks,
>
> Vaibhav Kulkarni
>
>