You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Matteo Alessandroni <sk...@apache.org> on 2019/03/14 16:52:30 UTC
Add a filter to intercept Knox requests and be able to add a custom
logic
Hi,
I'm trying to add a filter on a Knox instance so that I'll be able to
add a custom logic to (every / specific) REST requests to Knox.
I thought to use the "Class Path" feature [1], so I created a Maven Java
project, generated a ".jar" file and placed it in the
"$GATEWAY_HOME/ext" folder
Then, I thought it was necessary to link my custom class [2] on the
"gateway.xml" files of the resources I wanted, e.g.
./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
added:
<filter>
<role>rewrite</role>
<name>url-rewrite</name>
<class>com.test.knox.MyUrlRewriteServletFilter</class>
</filter>
then I made a REST request to Knox:
curl -i -k -u admin:admin-password -X GET
'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
but my filter is not called at all (I cannot see the log) and I'm not
sure whether it's because the class is not loaded or the filter is
placed in the wrong place or whatever.
On [4] you can see the project I have built.
So my questions are:
* what am I doing wrong or missing?
* is there a better way to do that? I just need to add a logic when
executing REST requests to Knox and make another REST call to an
external service I need.
Thank you!
Regards,
Matteo
[1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
[2]
https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
[3]
[4] https://github.com/mat-ale/apache-knox-filter
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Hi,
great, thank you so much for the help!
Best regards,
Matteo
On 29/03/19 17:22, Kevin Risden wrote:
> Should be able to use the default http client that is included in
> Knox. Should be accessible with getHttpClient() from
> AbstractGatewayDispatch.
>
> Kevin Risden
>
>
> On Fri, Mar 29, 2019 at 11:22 AM Matteo Alessandroni
> <skylark17@apache.org <ma...@apache.org>> wrote:
>
> Hi,
>
> Phil the "SubjectUtils.getCurrentEffectivePrincipalName()" method
> did the trick, thanks!
>
> Does anybody of you guys know how I could use a CXF REST client
> from my Dispatch class (that extends "DefaultDispatch") to call an
> external link?
> I tried to make a ".jar" that also includes the following dependencies
>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-frontend-jaxrs</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-rs-client</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-transports-http</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-transports-http-jetty</artifactId>
> </dependency>
>
> in order to execute:
>
> MyRequest request = new MyRequest();
> Client client = ClientBuilder.newBuilder().newClient();
> WebTarget target = client.target(MYURL);
> Invocation.Builder builder =
> target.request(MediaType.APPLICATION_JSON_TYPE);
> MyResponse response =
> builder.post(Entity.json(request.toString()),
> MyResponse.class);
>
> from the "executeRequest()" method:
>
> @Override
> protected void *executeRequest*(
> HttpUriRequest outboundRequest,
> HttpServletRequest inboundRequest,
> HttpServletResponse outboundResponse) throws IOException {
>
> but it does not work and it says:
>
> javax.ws.rs.ProcessingException: org.apache.cxf.BusException: No
> conduit initiator was found for the namespace
> http://cxf.apache.org/transports/http.
> at
> org.apache.cxf.jaxrs.client.AbstractClient.checkClientException(AbstractClient.java:638)
> at
> org.apache.cxf.jaxrs.client.AbstractClient.preProcessResult(AbstractClient.java:614)
> at
> org.apache.cxf.jaxrs.client.WebClient.doResponse(WebClient.java:1115)
> at
> org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1052)
> at
> org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:897)
> at
> org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:866)
> at
> org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:431)
> at
> org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:150)
> at
> org.apache.cxf.jaxrs.client.SyncInvokerImpl.post(SyncInvokerImpl.java:90)
> at
> org.apache.cxf.jaxrs.client.spec.InvocationBuilderImpl.post(InvocationBuilderImpl.java:158)
>
> is there something built-in I can use? Or maybe I'm doing
> something wrong with the CXF client?
>
> Thanks
>
>
>
> On 18/03/19 09:03, Matteo Alessandroni wrote:
>> Hi Phil,
>>
>> thank you for the help!
>> I'll check that class and will let you know.
>>
>> Thanks again!
>> Regards,
>> Matteo
>>
>>
>>
>> On 15/03/19 17:05, Phil Zampino wrote:
>>> I was going to suggest that you might be experiencing a docker
>>> issue because the ext folder works; You beat me to it.
>>>
>>> I'm not sure if this is what you're after, but
>>> org.apache.knox.gateway.security.SubjectUtils might be of use to
>>> you for identifying the currently logged-in principal.
>>>
>>> On Fri, Mar 15, 2019 at 11:53 AM Matteo Alessandroni
>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>
>>> Ok I think I found the issue:
>>> I had to restart the full Docker container to actually
>>> restart Knox.
>>> So I did that and it seems it's getting my class in "ext/" now.
>>>
>>> The error I get now is:
>>>
>>> Caused by:
>>> org.apache.knox.gateway.config.ConfigurationException:
>>> Failed to find configuration for provider as provider of
>>> org.apache.knox.gateway.dispatch.MyCustomDispatch via
>>> org.apache.knox.gateway.config.FilterConfigurationAdapter
>>>
>>> so I guess it's a problem with my class code.
>>> Is there a very simple code snippetI could use for my custom
>>> dispatch in order to access the "request" object in order to
>>> get the name of the user that is currently logged?
>>>
>>> Regards,
>>> Matteo
>>>
>>>
>>> On 15/03/19 15:49, Matteo Alessandroni wrote:
>>>> Thanks!
>>>> I checked that too, my ".jar" has the same permissions of
>>>> the other files in "lib/" folder (e.g.
>>>> "gateway-shell-1.2.0.jar").
>>>> I see your point about the script [1], but I also see that
>>>> the original gateway.sh (for Knox v1.2.0) [2] is not doing
>>>> anything special for the "ext/" folder, or am I wrong?
>>>>
>>>> Regards,
>>>> Matteo
>>>>
>>>>
>>>> [1]
>>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>>> [2]
>>>> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>>>>
>>>>
>>>> On 15/03/19 15:27, Sandeep Moré wrote:
>>>>> Ah, i missed the launcher.cfg, I see,
>>>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
>>>>> not take into account a lot of environmental variables and
>>>>> new stuff that is included in the
>>>>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
>>>>> that script is due for an update.
>>>>>
>>>>> The only other thing I can think of are the file
>>>>> permissions, make sure that your file can be read by knox.
>>>>>
>>>>> Best,
>>>>> Sandeep
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni
>>>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I tried also the "lib/" folder, but same problem.
>>>>> About the configuration for classpath yes it should be
>>>>> correct because I have the "bin/gateway.cfg" file
>>>>> containing:
>>>>>
>>>>> #Created from
>>>>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>>>>> #Thu Mar 14 10:10:31 GMT 2019
>>>>> GATEWAY_HOME=${launcher.dir}/..
>>>>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>>>>> <http://launcher.name>}-log4j.properties
>>>>> main.class=org.apache.knox.gateway.GatewayServer
>>>>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>>>>
>>>>> so it should load the ".jar" files in those folders.
>>>>>
>>>>> Is there anything I can do to debug this?
>>>>> FYI I'm using this Docker image [1] to run Knox and
>>>>> after adding files to "ext/" or "lib/" I use this
>>>>> script [2] to restart Knox and it seems to restart
>>>>> well, but still not seeing my classes in classpath.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> [1] https://github.com/moresandeep/knox-dev-docker
>>>>> [2]
>>>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>>>>
>>>>>
>>>>> On 15/03/19 14:56, Sandeep Moré wrote:
>>>>>> It should have picked it up, going through the
>>>>>> gateway.sh file, I don't see any place where ext
>>>>>> folder is added to class path, can you open a BUG for
>>>>>> this ?
>>>>>>
>>>>>> as a workaround for this you can copy your jar into
>>>>>> the lib folder and Knox should pick it up on the startup.
>>>>>>
>>>>>> Hopefully, this should help !
>>>>>>
>>>>>> Best,
>>>>>> Sandepe
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
>>>>>> <skylark17@apache.org <ma...@apache.org>>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Sandeep,
>>>>>>
>>>>>> thank you for your answer!
>>>>>> Ok so I tried to change my project and adding a
>>>>>> simple class like this:
>>>>>>
>>>>>> package com.test.ext;
>>>>>>
>>>>>> import java.io.IOException;
>>>>>> import java.net.URI;
>>>>>> import java.net.URISyntaxException;
>>>>>> import javax.servlet.http.HttpServletRequest;
>>>>>> import javax.servlet.http.HttpServletResponse;
>>>>>> import org.apache.knox.gateway.config.Configure;
>>>>>> import org.apache.knox.gateway.config.Default;
>>>>>> import
>>>>>> org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>>>>> import org.slf4j.Logger;
>>>>>> import org.slf4j.LoggerFactory;
>>>>>>
>>>>>> public class MyDispatch extends
>>>>>> AbstractGatewayDispatch {
>>>>>>
>>>>>> private static final Logger LOG =
>>>>>> LoggerFactory.getLogger(MyDispatch.class);
>>>>>>
>>>>>> @Override
>>>>>> public void destroy() {
>>>>>> LOG.debug("******* destroy()");
>>>>>> }
>>>>>>
>>>>>> @Configure
>>>>>> protected void customMethod(@Default("Test")
>>>>>> String test) {
>>>>>> LOG.debug("******* @Configure customMethod():
>>>>>> {}", test);
>>>>>> }
>>>>>>
>>>>>> @Override
>>>>>> public void doGet(URI url, HttpServletRequest
>>>>>> request, HttpServletResponse response)
>>>>>> throws IOException, URISyntaxException {
>>>>>>
>>>>>> LOG.debug("******* doGet() request: {}, {}",
>>>>>> request.getMethod(), new
>>>>>> URI(request.getRequestURI()));
>>>>>>
>>>>>> super.doGet(url, request, response);
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> made the ".jar" and put in the "ext" folder.
>>>>>> Executed the REST request again:
>>>>>>
>>>>>> curl -i -k -u admin:admin-password -X GET
>>>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>>
>>>>>> and I expected to see some logs, e.g. from the
>>>>>> "doGet()" method (I'm not sure about the
>>>>>> "@Configure" method, when should a method with
>>>>>> that annotation be executed?), but it seems it
>>>>>> does not see the class.
>>>>>> So I tried to configure my topology to use the
>>>>>> dispatch (as written on [1]):
>>>>>>
>>>>>> <service>
>>>>>> <role>WEBHDFS</role>
>>>>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>>>>
>>>>>> <dispatch>
>>>>>> <classname>com.test.ext.MyDispatch</classname>
>>>>>> <use-two-way-ssl>false</use-two-way-ssl>
>>>>>> </dispatch>
>>>>>> </service>
>>>>>>
>>>>>> and after saving logs say:
>>>>>>
>>>>>> ERROR knox.gateway
>>>>>> (GatewayFilter.java:doFilter(170)) - Gateway
>>>>>> processing failed:
>>>>>> javax.servlet.ServletException:
>>>>>> org.apache.shiro.subject.ExecutionException:
>>>>>> java.security.PrivilegedActionException:
>>>>>> javax.servlet.ServletException:
>>>>>> javax.servlet.ServletException:
>>>>>> java.lang.ClassNotFoundException:
>>>>>> com.test.ext.MyDispatch
>>>>>> javax.servlet.ServletException:
>>>>>> org.apache.shiro.subject.ExecutionException:
>>>>>> java.security.PrivilegedActionException:
>>>>>> javax.servlet.ServletException:
>>>>>> javax.servlet.ServletException:
>>>>>> *java.lang.ClassNotFoundException:
>>>>>> com.test.ext.MyDispatch*
>>>>>>
>>>>>> so it's not seeing my class.
>>>>>> What am I missing?
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>> Matteo
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>>>>
>>>>>>
>>>>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>>>>> Hello Matteo,
>>>>>>>
>>>>>>> I don't think this is a right way to add filters
>>>>>>> to Knox, if you want to add some custom logic
>>>>>>> between your Knox and the backend you can write
>>>>>>> custom dispatch [1]
>>>>>>> you can add this new jar in the ext folder and
>>>>>>> it should be picked up by Knox on start-up.
>>>>>>>
>>>>>>> [1]
>>>>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo
>>>>>>> Alessandroni <skylark17@apache.org
>>>>>>> <ma...@apache.org>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm trying to add a filter on a Knox
>>>>>>> instance so that I'll be able to add a
>>>>>>> custom logic to (every / specific) REST
>>>>>>> requests to Knox.
>>>>>>> I thought to use the "Class Path" feature
>>>>>>> [1], so I created a Maven Java project,
>>>>>>> generated a ".jar" file and placed it in the
>>>>>>> "$GATEWAY_HOME/ext" folder
>>>>>>> Then, I thought it was necessary to link my
>>>>>>> custom class [2] on the "gateway.xml" files
>>>>>>> of the resources I wanted, e.g.
>>>>>>>
>>>>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>>>>
>>>>>>> added:
>>>>>>>
>>>>>>> <filter>
>>>>>>> <role>rewrite</role>
>>>>>>> <name>url-rewrite</name>
>>>>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>>>>> </filter>
>>>>>>>
>>>>>>> then I made a REST request to Knox:
>>>>>>>
>>>>>>> curl -i -k -u admin:admin-password -X GET
>>>>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>>>
>>>>>>> but my filter is not called at all (I cannot
>>>>>>> see the log) and I'm not sure whether it's
>>>>>>> because the class is not loaded or the
>>>>>>> filter is placed in the wrong place or whatever.
>>>>>>>
>>>>>>> On [4] you can see the project I have built.
>>>>>>>
>>>>>>> So my questions are:
>>>>>>>
>>>>>>> * what am I doing wrong or missing?
>>>>>>> * is there a better way to do that? I just
>>>>>>> need to add a logic when executing REST
>>>>>>> requests to Knox and make another REST
>>>>>>> call to an external service I need.
>>>>>>>
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Regards,
>>>>>>> Matteo
>>>>>>>
>>>>>>> [1]
>>>>>>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>>>>> [2]
>>>>>>> [3]
>>>>>>> [4]
>>>>>>> https://github.com/mat-ale/apache-knox-filter
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Kevin Risden <kr...@apache.org>.
Should be able to use the default http client that is included in Knox.
Should be accessible with getHttpClient() from AbstractGatewayDispatch.
Kevin Risden
On Fri, Mar 29, 2019 at 11:22 AM Matteo Alessandroni <sk...@apache.org>
wrote:
> Hi,
>
> Phil the "SubjectUtils.getCurrentEffectivePrincipalName()" method did the
> trick, thanks!
>
> Does anybody of you guys know how I could use a CXF REST client from my
> Dispatch class (that extends "DefaultDispatch") to call an external link?
> I tried to make a ".jar" that also includes the following dependencies
>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-frontend-jaxrs</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-rs-client</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-transports-http</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.cxf</groupId>
> <artifactId>cxf-rt-transports-http-jetty</artifactId>
> </dependency>
>
> in order to execute:
>
> MyRequest request = new MyRequest();
> Client client = ClientBuilder.newBuilder().newClient();
> WebTarget target = client.target(MYURL);
> Invocation.Builder builder =
> target.request(MediaType.APPLICATION_JSON_TYPE);
> MyResponse response =
> builder.post(Entity.json(request.toString()), MyResponse.class);
>
> from the "executeRequest()" method:
>
> @Override
> protected void *executeRequest*(
> HttpUriRequest outboundRequest,
> HttpServletRequest inboundRequest,
> HttpServletResponse outboundResponse) throws IOException {
>
> but it does not work and it says:
>
> javax.ws.rs.ProcessingException: org.apache.cxf.BusException: No conduit
> initiator was found for the namespace
> http://cxf.apache.org/transports/http.
> at
> org.apache.cxf.jaxrs.client.AbstractClient.checkClientException(AbstractClient.java:638)
> at
> org.apache.cxf.jaxrs.client.AbstractClient.preProcessResult(AbstractClient.java:614)
> at
> org.apache.cxf.jaxrs.client.WebClient.doResponse(WebClient.java:1115)
> at
> org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1052)
> at
> org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:897)
> at
> org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:866)
> at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:431)
> at
> org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:150)
> at
> org.apache.cxf.jaxrs.client.SyncInvokerImpl.post(SyncInvokerImpl.java:90)
> at
> org.apache.cxf.jaxrs.client.spec.InvocationBuilderImpl.post(InvocationBuilderImpl.java:158)
>
> is there something built-in I can use? Or maybe I'm doing something wrong
> with the CXF client?
>
> Thanks
>
>
>
> On 18/03/19 09:03, Matteo Alessandroni wrote:
>
> Hi Phil,
>
> thank you for the help!
> I'll check that class and will let you know.
>
> Thanks again!
> Regards,
> Matteo
>
>
>
> On 15/03/19 17:05, Phil Zampino wrote:
>
> I was going to suggest that you might be experiencing a docker issue
> because the ext folder works; You beat me to it.
>
> I'm not sure if this is what you're after, but
> org.apache.knox.gateway.security.SubjectUtils might be of use to you for
> identifying the currently logged-in principal.
>
> On Fri, Mar 15, 2019 at 11:53 AM Matteo Alessandroni <sk...@apache.org>
> wrote:
>
>> Ok I think I found the issue:
>> I had to restart the full Docker container to actually restart Knox.
>> So I did that and it seems it's getting my class in "ext/" now.
>>
>> The error I get now is:
>>
>> Caused by: org.apache.knox.gateway.config.ConfigurationException: Failed
>> to find configuration for provider as provider of
>> org.apache.knox.gateway.dispatch.MyCustomDispatch via
>> org.apache.knox.gateway.config.FilterConfigurationAdapter
>>
>> so I guess it's a problem with my class code.
>> Is there a very simple code snippet I could use for my custom dispatch
>> in order to access the "request" object in order to get the name of the
>> user that is currently logged?
>>
>> Regards,
>> Matteo
>>
>>
>> On 15/03/19 15:49, Matteo Alessandroni wrote:
>>
>> Thanks!
>> I checked that too, my ".jar" has the same permissions of the other files
>> in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
>> I see your point about the script [1], but I also see that the original
>> gateway.sh (for Knox v1.2.0) [2] is not doing anything special for the
>> "ext/" folder, or am I wrong?
>>
>> Regards,
>> Matteo
>>
>>
>> [1]
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>> [2]
>> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>>
>>
>> On 15/03/19 15:27, Sandeep Moré wrote:
>>
>> Ah, i missed the launcher.cfg, I see,
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
>> not take into account a lot of environmental variables and new stuff that
>> is included in the
>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
>> that script is due for an update.
>>
>> The only other thing I can think of are the file permissions, make sure
>> that your file can be read by knox.
>>
>> Best,
>> Sandeep
>>
>>
>>
>> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni <
>> skylark17@apache.org> wrote:
>>
>>> Hi,
>>>
>>> I tried also the "lib/" folder, but same problem.
>>> About the configuration for classpath yes it should be correct because I
>>> have the "bin/gateway.cfg" file containing:
>>>
>>> #Created from
>>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>>> #Thu Mar 14 10:10:31 GMT 2019
>>> GATEWAY_HOME=${launcher.dir}/..
>>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>>> }-log4j.properties
>>> main.class=org.apache.knox.gateway.GatewayServer
>>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>>
>>> so it should load the ".jar" files in those folders.
>>>
>>> Is there anything I can do to debug this?
>>> FYI I'm using this Docker image [1] to run Knox and after adding files
>>> to "ext/" or "lib/" I use this script [2] to restart Knox and it seems to
>>> restart well, but still not seeing my classes in classpath.
>>>
>>> Thanks
>>>
>>>
>>> [1] https://github.com/moresandeep/knox-dev-docker
>>> [2]
>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>>
>>>
>>> On 15/03/19 14:56, Sandeep Moré wrote:
>>>
>>> It should have picked it up, going through the gateway.sh file, I don't
>>> see any place where ext folder is added to class path, can you open a BUG
>>> for this ?
>>>
>>> as a workaround for this you can copy your jar into the lib folder and
>>> Knox should pick it up on the startup.
>>>
>>> Hopefully, this should help !
>>>
>>> Best,
>>> Sandepe
>>>
>>>
>>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni <
>>> skylark17@apache.org> wrote:
>>>
>>>> Hi Sandeep,
>>>>
>>>> thank you for your answer!
>>>> Ok so I tried to change my project and adding a simple class like this:
>>>>
>>>> package com.test.ext;
>>>>
>>>> import java.io.IOException;
>>>> import java.net.URI;
>>>> import java.net.URISyntaxException;
>>>> import javax.servlet.http.HttpServletRequest;
>>>> import javax.servlet.http.HttpServletResponse;
>>>> import org.apache.knox.gateway.config.Configure;
>>>> import org.apache.knox.gateway.config.Default;
>>>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>>> import org.slf4j.Logger;
>>>> import org.slf4j.LoggerFactory;
>>>>
>>>> public class MyDispatch extends AbstractGatewayDispatch {
>>>>
>>>> private static final Logger LOG =
>>>> LoggerFactory.getLogger(MyDispatch.class);
>>>>
>>>> @Override
>>>> public void destroy() {
>>>> LOG.debug("******* destroy()");
>>>> }
>>>>
>>>> @Configure
>>>> protected void customMethod(@Default("Test") String test) {
>>>> LOG.debug("******* @Configure customMethod(): {}", test);
>>>> }
>>>>
>>>> @Override
>>>> public void doGet(URI url, HttpServletRequest request,
>>>> HttpServletResponse response)
>>>> throws IOException, URISyntaxException {
>>>>
>>>> LOG.debug("******* doGet() request: {}, {}",
>>>> request.getMethod(), new URI(request.getRequestURI()));
>>>>
>>>> super.doGet(url, request, response);
>>>> }
>>>>
>>>> }
>>>>
>>>> made the ".jar" and put in the "ext" folder.
>>>> Executed the REST request again:
>>>>
>>>> curl -i -k -u admin:admin-password -X GET '
>>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>
>>>> and I expected to see some logs, e.g. from the "doGet()" method (I'm
>>>> not sure about the "@Configure" method, when should a method with that
>>>> annotation be executed?), but it seems it does not see the class.
>>>> So I tried to configure my topology to use the dispatch (as written on
>>>> [1]):
>>>>
>>>> <service>
>>>> <role>WEBHDFS</role>
>>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>>
>>>> <dispatch>
>>>> <classname>com.test.ext.MyDispatch</classname>
>>>> <use-two-way-ssl>false</use-two-way-ssl>
>>>> </dispatch>
>>>> </service>
>>>>
>>>> and after saving logs say:
>>>>
>>>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
>>>> processing failed: javax.servlet.ServletException:
>>>> org.apache.shiro.subject.ExecutionException:
>>>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>>>> javax.servlet.ServletException: java.lang.ClassNotFoundException:
>>>> com.test.ext.MyDispatch
>>>> javax.servlet.ServletException:
>>>> org.apache.shiro.subject.ExecutionException:
>>>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>>>> javax.servlet.ServletException: *java.lang.ClassNotFoundException:
>>>> com.test.ext.MyDispatch*
>>>>
>>>> so it's not seeing my class.
>>>> What am I missing?
>>>>
>>>> Thank you!
>>>>
>>>> Matteo
>>>>
>>>>
>>>> [1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>>
>>>>
>>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>>
>>>> Hello Matteo,
>>>>
>>>> I don't think this is a right way to add filters to Knox, if you want
>>>> to add some custom logic between your Knox and the backend you can write
>>>> custom dispatch [1]
>>>> you can add this new jar in the ext folder and it should be picked up
>>>> by Knox on start-up.
>>>>
>>>> [1]
>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>>
>>>>
>>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <
>>>> skylark17@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm trying to add a filter on a Knox instance so that I'll be able to
>>>>> add a custom logic to (every / specific) REST requests to Knox.
>>>>> I thought to use the "Class Path" feature [1], so I created a Maven
>>>>> Java project, generated a ".jar" file and placed it in the "
>>>>> $GATEWAY_HOME/ext" folder
>>>>> Then, I thought it was necessary to link my custom class [2] on the
>>>>> "gateway.xml" files of the resources I wanted, e.g.
>>>>>
>>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>>
>>>>> added:
>>>>>
>>>>> <filter>
>>>>> <role>rewrite</role>
>>>>> <name>url-rewrite</name>
>>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>>> </filter>
>>>>>
>>>>> then I made a REST request to Knox:
>>>>>
>>>>> curl -i -k -u admin:admin-password -X GET '
>>>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>
>>>>> but my filter is not called at all (I cannot see the log) and I'm not
>>>>> sure whether it's because the class is not loaded or the filter is placed
>>>>> in the wrong place or whatever.
>>>>>
>>>>> On [4] you can see the project I have built.
>>>>>
>>>>> So my questions are:
>>>>>
>>>>> - what am I doing wrong or missing?
>>>>> - is there a better way to do that? I just need to add a logic
>>>>> when executing REST requests to Knox and make another REST call to an
>>>>> external service I need.
>>>>>
>>>>>
>>>>> Thank you!
>>>>> Regards,
>>>>> Matteo
>>>>>
>>>>> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>>> [2]
>>>>> [3]
>>>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Hi,
Phil the "SubjectUtils.getCurrentEffectivePrincipalName()" method did
the trick, thanks!
Does anybody of you guys know how I could use a CXF REST client from my
Dispatch class (that extends "DefaultDispatch") to call an external link?
I tried to make a ".jar" that also includes the following dependencies
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxrs</artifactId>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-rs-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http-jetty</artifactId>
</dependency>
in order to execute:
MyRequest request = new MyRequest();
Client client = ClientBuilder.newBuilder().newClient();
WebTarget target = client.target(MYURL);
Invocation.Builder builder =
target.request(MediaType.APPLICATION_JSON_TYPE);
MyResponse response =
builder.post(Entity.json(request.toString()), MyResponse.class);
from the "executeRequest()" method:
@Override
protected void *executeRequest*(
HttpUriRequest outboundRequest,
HttpServletRequest inboundRequest,
HttpServletResponse outboundResponse) throws IOException {
but it does not work and it says:
javax.ws.rs.ProcessingException: org.apache.cxf.BusException: No conduit
initiator was found for the namespace http://cxf.apache.org/transports/http.
at
org.apache.cxf.jaxrs.client.AbstractClient.checkClientException(AbstractClient.java:638)
at
org.apache.cxf.jaxrs.client.AbstractClient.preProcessResult(AbstractClient.java:614)
at
org.apache.cxf.jaxrs.client.WebClient.doResponse(WebClient.java:1115)
at
org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1052)
at
org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:897)
at
org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:866)
at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:431)
at
org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:150)
at
org.apache.cxf.jaxrs.client.SyncInvokerImpl.post(SyncInvokerImpl.java:90)
at
org.apache.cxf.jaxrs.client.spec.InvocationBuilderImpl.post(InvocationBuilderImpl.java:158)
is there something built-in I can use? Or maybe I'm doing something
wrong with the CXF client?
Thanks
On 18/03/19 09:03, Matteo Alessandroni wrote:
> Hi Phil,
>
> thank you for the help!
> I'll check that class and will let you know.
>
> Thanks again!
> Regards,
> Matteo
>
>
>
> On 15/03/19 17:05, Phil Zampino wrote:
>> I was going to suggest that you might be experiencing a docker issue
>> because the ext folder works; You beat me to it.
>>
>> I'm not sure if this is what you're after, but
>> org.apache.knox.gateway.security.SubjectUtils might be of use to you
>> for identifying the currently logged-in principal.
>>
>> On Fri, Mar 15, 2019 at 11:53 AM Matteo Alessandroni
>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>
>> Ok I think I found the issue:
>> I had to restart the full Docker container to actually restart Knox.
>> So I did that and it seems it's getting my class in "ext/" now.
>>
>> The error I get now is:
>>
>> Caused by: org.apache.knox.gateway.config.ConfigurationException:
>> Failed to find configuration for provider as provider of
>> org.apache.knox.gateway.dispatch.MyCustomDispatch via
>> org.apache.knox.gateway.config.FilterConfigurationAdapter
>>
>> so I guess it's a problem with my class code.
>> Is there a very simple code snippetI could use for my custom
>> dispatch in order to access the "request" object in order to get
>> the name of the user that is currently logged?
>>
>> Regards,
>> Matteo
>>
>>
>> On 15/03/19 15:49, Matteo Alessandroni wrote:
>>> Thanks!
>>> I checked that too, my ".jar" has the same permissions of the
>>> other files in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
>>> I see your point about the script [1], but I also see that the
>>> original gateway.sh (for Knox v1.2.0) [2] is not doing anything
>>> special for the "ext/" folder, or am I wrong?
>>>
>>> Regards,
>>> Matteo
>>>
>>>
>>> [1]
>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>> [2]
>>> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>>>
>>>
>>> On 15/03/19 15:27, Sandeep Moré wrote:
>>>> Ah, i missed the launcher.cfg, I see,
>>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
>>>> not take into account a lot of environmental variables and new
>>>> stuff that is included in the
>>>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
>>>> that script is due for an update.
>>>>
>>>> The only other thing I can think of are the file permissions,
>>>> make sure that your file can be read by knox.
>>>>
>>>> Best,
>>>> Sandeep
>>>>
>>>>
>>>>
>>>> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni
>>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I tried also the "lib/" folder, but same problem.
>>>> About the configuration for classpath yes it should be
>>>> correct because I have the "bin/gateway.cfg" file containing:
>>>>
>>>> #Created from
>>>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>>>> #Thu Mar 14 10:10:31 GMT 2019
>>>> GATEWAY_HOME=${launcher.dir}/..
>>>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>>>> <http://launcher.name>}-log4j.properties
>>>> main.class=org.apache.knox.gateway.GatewayServer
>>>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>>>
>>>> so it should load the ".jar" files in those folders.
>>>>
>>>> Is there anything I can do to debug this?
>>>> FYI I'm using this Docker image [1] to run Knox and after
>>>> adding files to "ext/" or "lib/" I use this script [2] to
>>>> restart Knox and it seems to restart well, but still not
>>>> seeing my classes in classpath.
>>>>
>>>> Thanks
>>>>
>>>>
>>>> [1] https://github.com/moresandeep/knox-dev-docker
>>>> [2]
>>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>>>
>>>>
>>>> On 15/03/19 14:56, Sandeep Moré wrote:
>>>>> It should have picked it up, going through the gateway.sh
>>>>> file, I don't see any place where ext folder is added to
>>>>> class path, can you open a BUG for this ?
>>>>>
>>>>> as a workaround for this you can copy your jar into the
>>>>> lib folder and Knox should pick it up on the startup.
>>>>>
>>>>> Hopefully, this should help !
>>>>>
>>>>> Best,
>>>>> Sandepe
>>>>>
>>>>>
>>>>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
>>>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>>>
>>>>> Hi Sandeep,
>>>>>
>>>>> thank you for your answer!
>>>>> Ok so I tried to change my project and adding a simple
>>>>> class like this:
>>>>>
>>>>> package com.test.ext;
>>>>>
>>>>> import java.io.IOException;
>>>>> import java.net.URI;
>>>>> import java.net.URISyntaxException;
>>>>> import javax.servlet.http.HttpServletRequest;
>>>>> import javax.servlet.http.HttpServletResponse;
>>>>> import org.apache.knox.gateway.config.Configure;
>>>>> import org.apache.knox.gateway.config.Default;
>>>>> import
>>>>> org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>>>> import org.slf4j.Logger;
>>>>> import org.slf4j.LoggerFactory;
>>>>>
>>>>> public class MyDispatch extends AbstractGatewayDispatch {
>>>>>
>>>>> private static final Logger LOG =
>>>>> LoggerFactory.getLogger(MyDispatch.class);
>>>>>
>>>>> @Override
>>>>> public void destroy() {
>>>>> LOG.debug("******* destroy()");
>>>>> }
>>>>>
>>>>> @Configure
>>>>> protected void customMethod(@Default("Test")
>>>>> String test) {
>>>>> LOG.debug("******* @Configure customMethod():
>>>>> {}", test);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public void doGet(URI url, HttpServletRequest
>>>>> request, HttpServletResponse response)
>>>>> throws IOException, URISyntaxException {
>>>>>
>>>>> LOG.debug("******* doGet() request: {}, {}",
>>>>> request.getMethod(), new URI(request.getRequestURI()));
>>>>>
>>>>> super.doGet(url, request, response);
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> made the ".jar" and put in the "ext" folder.
>>>>> Executed the REST request again:
>>>>>
>>>>> curl -i -k -u admin:admin-password -X GET
>>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>
>>>>> and I expected to see some logs, e.g. from the
>>>>> "doGet()" method (I'm not sure about the "@Configure"
>>>>> method, when should a method with that annotation be
>>>>> executed?), but it seems it does not see the class.
>>>>> So I tried to configure my topology to use the
>>>>> dispatch (as written on [1]):
>>>>>
>>>>> <service>
>>>>> <role>WEBHDFS</role>
>>>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>>>
>>>>> <dispatch>
>>>>> <classname>com.test.ext.MyDispatch</classname>
>>>>> <use-two-way-ssl>false</use-two-way-ssl>
>>>>> </dispatch>
>>>>> </service>
>>>>>
>>>>> and after saving logs say:
>>>>>
>>>>> ERROR knox.gateway (GatewayFilter.java:doFilter(170))
>>>>> - Gateway processing failed:
>>>>> javax.servlet.ServletException:
>>>>> org.apache.shiro.subject.ExecutionException:
>>>>> java.security.PrivilegedActionException:
>>>>> javax.servlet.ServletException:
>>>>> javax.servlet.ServletException:
>>>>> java.lang.ClassNotFoundException: com.test.ext.MyDispatch
>>>>> javax.servlet.ServletException:
>>>>> org.apache.shiro.subject.ExecutionException:
>>>>> java.security.PrivilegedActionException:
>>>>> javax.servlet.ServletException:
>>>>> javax.servlet.ServletException:
>>>>> *java.lang.ClassNotFoundException:
>>>>> com.test.ext.MyDispatch*
>>>>>
>>>>> so it's not seeing my class.
>>>>> What am I missing?
>>>>>
>>>>> Thank you!
>>>>>
>>>>> Matteo
>>>>>
>>>>>
>>>>> [1]
>>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>>>
>>>>>
>>>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>>>> Hello Matteo,
>>>>>>
>>>>>> I don't think this is a right way to add filters to
>>>>>> Knox, if you want to add some custom logic between
>>>>>> your Knox and the backend you can write custom
>>>>>> dispatch [1]
>>>>>> you can add this new jar in the ext folder and it
>>>>>> should be picked up by Knox on start-up.
>>>>>>
>>>>>> [1]
>>>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
>>>>>> <skylark17@apache.org <ma...@apache.org>>
>>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm trying to add a filter on a Knox instance so
>>>>>> that I'll be able to add a custom logic to (every
>>>>>> / specific) REST requests to Knox.
>>>>>> I thought to use the "Class Path" feature [1], so
>>>>>> I created a Maven Java project, generated a
>>>>>> ".jar" file and placed it in the
>>>>>> "$GATEWAY_HOME/ext" folder
>>>>>> Then, I thought it was necessary to link my
>>>>>> custom class [2] on the "gateway.xml" files of
>>>>>> the resources I wanted, e.g.
>>>>>>
>>>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>>>
>>>>>> added:
>>>>>>
>>>>>> <filter>
>>>>>> <role>rewrite</role>
>>>>>> <name>url-rewrite</name>
>>>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>>>> </filter>
>>>>>>
>>>>>> then I made a REST request to Knox:
>>>>>>
>>>>>> curl -i -k -u admin:admin-password -X GET
>>>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>>
>>>>>> but my filter is not called at all (I cannot see
>>>>>> the log) and I'm not sure whether it's because
>>>>>> the class is not loaded or the filter is placed
>>>>>> in the wrong place or whatever.
>>>>>>
>>>>>> On [4] you can see the project I have built.
>>>>>>
>>>>>> So my questions are:
>>>>>>
>>>>>> * what am I doing wrong or missing?
>>>>>> * is there a better way to do that? I just need
>>>>>> to add a logic when executing REST requests
>>>>>> to Knox and make another REST call to an
>>>>>> external service I need.
>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>> Regards,
>>>>>> Matteo
>>>>>>
>>>>>> [1]
>>>>>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>>>> [2]
>>>>>> [3]
>>>>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Hi Phil,
thank you for the help!
I'll check that class and will let you know.
Thanks again!
Regards,
Matteo
On 15/03/19 17:05, Phil Zampino wrote:
> I was going to suggest that you might be experiencing a docker issue
> because the ext folder works; You beat me to it.
>
> I'm not sure if this is what you're after, but
> org.apache.knox.gateway.security.SubjectUtils might be of use to you
> for identifying the currently logged-in principal.
>
> On Fri, Mar 15, 2019 at 11:53 AM Matteo Alessandroni
> <skylark17@apache.org <ma...@apache.org>> wrote:
>
> Ok I think I found the issue:
> I had to restart the full Docker container to actually restart Knox.
> So I did that and it seems it's getting my class in "ext/" now.
>
> The error I get now is:
>
> Caused by: org.apache.knox.gateway.config.ConfigurationException:
> Failed to find configuration for provider as provider of
> org.apache.knox.gateway.dispatch.MyCustomDispatch via
> org.apache.knox.gateway.config.FilterConfigurationAdapter
>
> so I guess it's a problem with my class code.
> Is there a very simple code snippetI could use for my custom
> dispatch in order to access the "request" object in order to get
> the name of the user that is currently logged?
>
> Regards,
> Matteo
>
>
> On 15/03/19 15:49, Matteo Alessandroni wrote:
>> Thanks!
>> I checked that too, my ".jar" has the same permissions of the
>> other files in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
>> I see your point about the script [1], but I also see that the
>> original gateway.sh (for Knox v1.2.0) [2] is not doing anything
>> special for the "ext/" folder, or am I wrong?
>>
>> Regards,
>> Matteo
>>
>>
>> [1]
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>> [2]
>> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>>
>>
>> On 15/03/19 15:27, Sandeep Moré wrote:
>>> Ah, i missed the launcher.cfg, I see,
>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
>>> not take into account a lot of environmental variables and new
>>> stuff that is included in the
>>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
>>> that script is due for an update.
>>>
>>> The only other thing I can think of are the file permissions,
>>> make sure that your file can be read by knox.
>>>
>>> Best,
>>> Sandeep
>>>
>>>
>>>
>>> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni
>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>
>>> Hi,
>>>
>>> I tried also the "lib/" folder, but same problem.
>>> About the configuration for classpath yes it should be
>>> correct because I have the "bin/gateway.cfg" file containing:
>>>
>>> #Created from
>>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>>> #Thu Mar 14 10:10:31 GMT 2019
>>> GATEWAY_HOME=${launcher.dir}/..
>>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>>> <http://launcher.name>}-log4j.properties
>>> main.class=org.apache.knox.gateway.GatewayServer
>>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>>
>>> so it should load the ".jar" files in those folders.
>>>
>>> Is there anything I can do to debug this?
>>> FYI I'm using this Docker image [1] to run Knox and after
>>> adding files to "ext/" or "lib/" I use this script [2] to
>>> restart Knox and it seems to restart well, but still not
>>> seeing my classes in classpath.
>>>
>>> Thanks
>>>
>>>
>>> [1] https://github.com/moresandeep/knox-dev-docker
>>> [2]
>>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>>
>>>
>>> On 15/03/19 14:56, Sandeep Moré wrote:
>>>> It should have picked it up, going through the gateway.sh
>>>> file, I don't see any place where ext folder is added to
>>>> class path, can you open a BUG for this ?
>>>>
>>>> as a workaround for this you can copy your jar into the lib
>>>> folder and Knox should pick it up on the startup.
>>>>
>>>> Hopefully, this should help !
>>>>
>>>> Best,
>>>> Sandepe
>>>>
>>>>
>>>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
>>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>>
>>>> Hi Sandeep,
>>>>
>>>> thank you for your answer!
>>>> Ok so I tried to change my project and adding a simple
>>>> class like this:
>>>>
>>>> package com.test.ext;
>>>>
>>>> import java.io.IOException;
>>>> import java.net.URI;
>>>> import java.net.URISyntaxException;
>>>> import javax.servlet.http.HttpServletRequest;
>>>> import javax.servlet.http.HttpServletResponse;
>>>> import org.apache.knox.gateway.config.Configure;
>>>> import org.apache.knox.gateway.config.Default;
>>>> import
>>>> org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>>> import org.slf4j.Logger;
>>>> import org.slf4j.LoggerFactory;
>>>>
>>>> public class MyDispatch extends AbstractGatewayDispatch {
>>>>
>>>> private static final Logger LOG =
>>>> LoggerFactory.getLogger(MyDispatch.class);
>>>>
>>>> @Override
>>>> public void destroy() {
>>>> LOG.debug("******* destroy()");
>>>> }
>>>>
>>>> @Configure
>>>> protected void customMethod(@Default("Test") String
>>>> test) {
>>>> LOG.debug("******* @Configure customMethod():
>>>> {}", test);
>>>> }
>>>>
>>>> @Override
>>>> public void doGet(URI url, HttpServletRequest
>>>> request, HttpServletResponse response)
>>>> throws IOException, URISyntaxException {
>>>>
>>>> LOG.debug("******* doGet() request: {}, {}",
>>>> request.getMethod(), new URI(request.getRequestURI()));
>>>>
>>>> super.doGet(url, request, response);
>>>> }
>>>>
>>>> }
>>>>
>>>> made the ".jar" and put in the "ext" folder.
>>>> Executed the REST request again:
>>>>
>>>> curl -i -k -u admin:admin-password -X GET
>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>
>>>> and I expected to see some logs, e.g. from the
>>>> "doGet()" method (I'm not sure about the "@Configure"
>>>> method, when should a method with that annotation be
>>>> executed?), but it seems it does not see the class.
>>>> So I tried to configure my topology to use the dispatch
>>>> (as written on [1]):
>>>>
>>>> <service>
>>>> <role>WEBHDFS</role>
>>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>>
>>>> <dispatch>
>>>> <classname>com.test.ext.MyDispatch</classname>
>>>> <use-two-way-ssl>false</use-two-way-ssl>
>>>> </dispatch>
>>>> </service>
>>>>
>>>> and after saving logs say:
>>>>
>>>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) -
>>>> Gateway processing failed:
>>>> javax.servlet.ServletException:
>>>> org.apache.shiro.subject.ExecutionException:
>>>> java.security.PrivilegedActionException:
>>>> javax.servlet.ServletException:
>>>> javax.servlet.ServletException:
>>>> java.lang.ClassNotFoundException: com.test.ext.MyDispatch
>>>> javax.servlet.ServletException:
>>>> org.apache.shiro.subject.ExecutionException:
>>>> java.security.PrivilegedActionException:
>>>> javax.servlet.ServletException:
>>>> javax.servlet.ServletException:
>>>> *java.lang.ClassNotFoundException: com.test.ext.MyDispatch*
>>>>
>>>> so it's not seeing my class.
>>>> What am I missing?
>>>>
>>>> Thank you!
>>>>
>>>> Matteo
>>>>
>>>>
>>>> [1]
>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>>
>>>>
>>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>>> Hello Matteo,
>>>>>
>>>>> I don't think this is a right way to add filters to
>>>>> Knox, if you want to add some custom logic between
>>>>> your Knox and the backend you can write custom
>>>>> dispatch [1]
>>>>> you can add this new jar in the ext folder and it
>>>>> should be picked up by Knox on start-up.
>>>>>
>>>>> [1]
>>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>>>
>>>>>
>>>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
>>>>> <skylark17@apache.org <ma...@apache.org>>
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm trying to add a filter on a Knox instance so
>>>>> that I'll be able to add a custom logic to (every
>>>>> / specific) REST requests to Knox.
>>>>> I thought to use the "Class Path" feature [1], so
>>>>> I created a Maven Java project, generated a ".jar"
>>>>> file and placed it in the "$GATEWAY_HOME/ext" folder
>>>>> Then, I thought it was necessary to link my custom
>>>>> class [2] on the "gateway.xml" files of the
>>>>> resources I wanted, e.g.
>>>>>
>>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>>
>>>>> added:
>>>>>
>>>>> <filter>
>>>>> <role>rewrite</role>
>>>>> <name>url-rewrite</name>
>>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>>> </filter>
>>>>>
>>>>> then I made a REST request to Knox:
>>>>>
>>>>> curl -i -k -u admin:admin-password -X GET
>>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>>
>>>>> but my filter is not called at all (I cannot see
>>>>> the log) and I'm not sure whether it's because the
>>>>> class is not loaded or the filter is placed in the
>>>>> wrong place or whatever.
>>>>>
>>>>> On [4] you can see the project I have built.
>>>>>
>>>>> So my questions are:
>>>>>
>>>>> * what am I doing wrong or missing?
>>>>> * is there a better way to do that? I just need
>>>>> to add a logic when executing REST requests to
>>>>> Knox and make another REST call to an external
>>>>> service I need.
>>>>>
>>>>>
>>>>> Thank you!
>>>>> Regards,
>>>>> Matteo
>>>>>
>>>>> [1]
>>>>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>>> [2]
>>>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>>>> [3]
>>>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Phil Zampino <pz...@apache.org>.
I was going to suggest that you might be experiencing a docker issue
because the ext folder works; You beat me to it.
I'm not sure if this is what you're after, but
org.apache.knox.gateway.security.SubjectUtils might be of use to you for
identifying the currently logged-in principal.
On Fri, Mar 15, 2019 at 11:53 AM Matteo Alessandroni <sk...@apache.org>
wrote:
> Ok I think I found the issue:
> I had to restart the full Docker container to actually restart Knox.
> So I did that and it seems it's getting my class in "ext/" now.
>
> The error I get now is:
>
> Caused by: org.apache.knox.gateway.config.ConfigurationException: Failed
> to find configuration for provider as provider of
> org.apache.knox.gateway.dispatch.MyCustomDispatch via
> org.apache.knox.gateway.config.FilterConfigurationAdapter
>
> so I guess it's a problem with my class code.
> Is there a very simple code snippet I could use for my custom dispatch in
> order to access the "request" object in order to get the name of the user
> that is currently logged?
>
> Regards,
> Matteo
>
>
> On 15/03/19 15:49, Matteo Alessandroni wrote:
>
> Thanks!
> I checked that too, my ".jar" has the same permissions of the other files
> in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
> I see your point about the script [1], but I also see that the original
> gateway.sh (for Knox v1.2.0) [2] is not doing anything special for the
> "ext/" folder, or am I wrong?
>
> Regards,
> Matteo
>
>
> [1]
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
> [2]
> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>
>
> On 15/03/19 15:27, Sandeep Moré wrote:
>
> Ah, i missed the launcher.cfg, I see,
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
> not take into account a lot of environmental variables and new stuff that
> is included in the
> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
> that script is due for an update.
>
> The only other thing I can think of are the file permissions, make sure
> that your file can be read by knox.
>
> Best,
> Sandeep
>
>
>
> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni <sk...@apache.org>
> wrote:
>
>> Hi,
>>
>> I tried also the "lib/" folder, but same problem.
>> About the configuration for classpath yes it should be correct because I
>> have the "bin/gateway.cfg" file containing:
>>
>> #Created from
>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>> #Thu Mar 14 10:10:31 GMT 2019
>> GATEWAY_HOME=${launcher.dir}/..
>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>> }-log4j.properties
>> main.class=org.apache.knox.gateway.GatewayServer
>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>
>> so it should load the ".jar" files in those folders.
>>
>> Is there anything I can do to debug this?
>> FYI I'm using this Docker image [1] to run Knox and after adding files
>> to "ext/" or "lib/" I use this script [2] to restart Knox and it seems to
>> restart well, but still not seeing my classes in classpath.
>>
>> Thanks
>>
>>
>> [1] https://github.com/moresandeep/knox-dev-docker
>> [2]
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>
>>
>> On 15/03/19 14:56, Sandeep Moré wrote:
>>
>> It should have picked it up, going through the gateway.sh file, I don't
>> see any place where ext folder is added to class path, can you open a BUG
>> for this ?
>>
>> as a workaround for this you can copy your jar into the lib folder and
>> Knox should pick it up on the startup.
>>
>> Hopefully, this should help !
>>
>> Best,
>> Sandepe
>>
>>
>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni <sk...@apache.org>
>> wrote:
>>
>>> Hi Sandeep,
>>>
>>> thank you for your answer!
>>> Ok so I tried to change my project and adding a simple class like this:
>>>
>>> package com.test.ext;
>>>
>>> import java.io.IOException;
>>> import java.net.URI;
>>> import java.net.URISyntaxException;
>>> import javax.servlet.http.HttpServletRequest;
>>> import javax.servlet.http.HttpServletResponse;
>>> import org.apache.knox.gateway.config.Configure;
>>> import org.apache.knox.gateway.config.Default;
>>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>> import org.slf4j.Logger;
>>> import org.slf4j.LoggerFactory;
>>>
>>> public class MyDispatch extends AbstractGatewayDispatch {
>>>
>>> private static final Logger LOG =
>>> LoggerFactory.getLogger(MyDispatch.class);
>>>
>>> @Override
>>> public void destroy() {
>>> LOG.debug("******* destroy()");
>>> }
>>>
>>> @Configure
>>> protected void customMethod(@Default("Test") String test) {
>>> LOG.debug("******* @Configure customMethod(): {}", test);
>>> }
>>>
>>> @Override
>>> public void doGet(URI url, HttpServletRequest request,
>>> HttpServletResponse response)
>>> throws IOException, URISyntaxException {
>>>
>>> LOG.debug("******* doGet() request: {}, {}",
>>> request.getMethod(), new URI(request.getRequestURI()));
>>>
>>> super.doGet(url, request, response);
>>> }
>>>
>>> }
>>>
>>> made the ".jar" and put in the "ext" folder.
>>> Executed the REST request again:
>>>
>>> curl -i -k -u admin:admin-password -X GET '
>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>
>>> and I expected to see some logs, e.g. from the "doGet()" method (I'm not
>>> sure about the "@Configure" method, when should a method with that
>>> annotation be executed?), but it seems it does not see the class.
>>> So I tried to configure my topology to use the dispatch (as written on
>>> [1]):
>>>
>>> <service>
>>> <role>WEBHDFS</role>
>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>
>>> <dispatch>
>>> <classname>com.test.ext.MyDispatch</classname>
>>> <use-two-way-ssl>false</use-two-way-ssl>
>>> </dispatch>
>>> </service>
>>>
>>> and after saving logs say:
>>>
>>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
>>> processing failed: javax.servlet.ServletException:
>>> org.apache.shiro.subject.ExecutionException:
>>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>>> javax.servlet.ServletException: java.lang.ClassNotFoundException:
>>> com.test.ext.MyDispatch
>>> javax.servlet.ServletException:
>>> org.apache.shiro.subject.ExecutionException:
>>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>>> javax.servlet.ServletException: *java.lang.ClassNotFoundException:
>>> com.test.ext.MyDispatch*
>>>
>>> so it's not seeing my class.
>>> What am I missing?
>>>
>>> Thank you!
>>>
>>> Matteo
>>>
>>>
>>> [1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>
>>>
>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>
>>> Hello Matteo,
>>>
>>> I don't think this is a right way to add filters to Knox, if you want to
>>> add some custom logic between your Knox and the backend you can write
>>> custom dispatch [1]
>>> you can add this new jar in the ext folder and it should be picked up by
>>> Knox on start-up.
>>>
>>> [1]
>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>
>>>
>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <
>>> skylark17@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm trying to add a filter on a Knox instance so that I'll be able to
>>>> add a custom logic to (every / specific) REST requests to Knox.
>>>> I thought to use the "Class Path" feature [1], so I created a Maven
>>>> Java project, generated a ".jar" file and placed it in the "
>>>> $GATEWAY_HOME/ext" folder
>>>> Then, I thought it was necessary to link my custom class [2] on the
>>>> "gateway.xml" files of the resources I wanted, e.g.
>>>>
>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>
>>>> added:
>>>>
>>>> <filter>
>>>> <role>rewrite</role>
>>>> <name>url-rewrite</name>
>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>> </filter>
>>>>
>>>> then I made a REST request to Knox:
>>>>
>>>> curl -i -k -u admin:admin-password -X GET '
>>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>
>>>> but my filter is not called at all (I cannot see the log) and I'm not
>>>> sure whether it's because the class is not loaded or the filter is placed
>>>> in the wrong place or whatever.
>>>>
>>>> On [4] you can see the project I have built.
>>>>
>>>> So my questions are:
>>>>
>>>> - what am I doing wrong or missing?
>>>> - is there a better way to do that? I just need to add a logic when
>>>> executing REST requests to Knox and make another REST call to an external
>>>> service I need.
>>>>
>>>>
>>>> Thank you!
>>>> Regards,
>>>> Matteo
>>>>
>>>> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>> [2]
>>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>>> [3]
>>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Ok I think I found the issue:
I had to restart the full Docker container to actually restart Knox.
So I did that and it seems it's getting my class in "ext/" now.
The error I get now is:
Caused by: org.apache.knox.gateway.config.ConfigurationException: Failed
to find configuration for provider as provider of
org.apache.knox.gateway.dispatch.MyCustomDispatch via
org.apache.knox.gateway.config.FilterConfigurationAdapter
so I guess it's a problem with my class code.
Is there a very simple code snippetI could use for my custom dispatch in
order to access the "request" object in order to get the name of the
user that is currently logged?
Regards,
Matteo
On 15/03/19 15:49, Matteo Alessandroni wrote:
> Thanks!
> I checked that too, my ".jar" has the same permissions of the other
> files in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
> I see your point about the script [1], but I also see that the
> original gateway.sh (for Knox v1.2.0) [2] is not doing anything
> special for the "ext/" folder, or am I wrong?
>
> Regards,
> Matteo
>
>
> [1]
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
> [2]
> https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
>
>
> On 15/03/19 15:27, Sandeep Moré wrote:
>> Ah, i missed the launcher.cfg, I see,
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
>> not take into account a lot of environmental variables and new stuff
>> that is included in the
>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
>> that script is due for an update.
>>
>> The only other thing I can think of are the file permissions, make
>> sure that your file can be read by knox.
>>
>> Best,
>> Sandeep
>>
>>
>>
>> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni
>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>
>> Hi,
>>
>> I tried also the "lib/" folder, but same problem.
>> About the configuration for classpath yes it should be correct
>> because I have the "bin/gateway.cfg" file containing:
>>
>> #Created from
>> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
>> #Thu Mar 14 10:10:31 GMT 2019
>> GATEWAY_HOME=${launcher.dir}/..
>> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
>> <http://launcher.name>}-log4j.properties
>> main.class=org.apache.knox.gateway.GatewayServer
>> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>>
>> so it should load the ".jar" files in those folders.
>>
>> Is there anything I can do to debug this?
>> FYI I'm using this Docker image [1] to run Knox and after adding
>> files to "ext/" or "lib/" I use this script [2] to restart Knox
>> and it seems to restart well, but still not seeing my classes in
>> classpath.
>>
>> Thanks
>>
>>
>> [1] https://github.com/moresandeep/knox-dev-docker
>> [2]
>> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>>
>>
>> On 15/03/19 14:56, Sandeep Moré wrote:
>>> It should have picked it up, going through the gateway.sh file,
>>> I don't see any place where ext folder is added to class path,
>>> can you open a BUG for this ?
>>>
>>> as a workaround for this you can copy your jar into the lib
>>> folder and Knox should pick it up on the startup.
>>>
>>> Hopefully, this should help !
>>>
>>> Best,
>>> Sandepe
>>>
>>>
>>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>
>>> Hi Sandeep,
>>>
>>> thank you for your answer!
>>> Ok so I tried to change my project and adding a simple class
>>> like this:
>>>
>>> package com.test.ext;
>>>
>>> import java.io.IOException;
>>> import java.net.URI;
>>> import java.net.URISyntaxException;
>>> import javax.servlet.http.HttpServletRequest;
>>> import javax.servlet.http.HttpServletResponse;
>>> import org.apache.knox.gateway.config.Configure;
>>> import org.apache.knox.gateway.config.Default;
>>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>>> import org.slf4j.Logger;
>>> import org.slf4j.LoggerFactory;
>>>
>>> public class MyDispatch extends AbstractGatewayDispatch {
>>>
>>> private static final Logger LOG =
>>> LoggerFactory.getLogger(MyDispatch.class);
>>>
>>> @Override
>>> public void destroy() {
>>> LOG.debug("******* destroy()");
>>> }
>>>
>>> @Configure
>>> protected void customMethod(@Default("Test") String test) {
>>> LOG.debug("******* @Configure customMethod(): {}",
>>> test);
>>> }
>>>
>>> @Override
>>> public void doGet(URI url, HttpServletRequest request,
>>> HttpServletResponse response)
>>> throws IOException, URISyntaxException {
>>>
>>> LOG.debug("******* doGet() request: {}, {}",
>>> request.getMethod(), new URI(request.getRequestURI()));
>>>
>>> super.doGet(url, request, response);
>>> }
>>>
>>> }
>>>
>>> made the ".jar" and put in the "ext" folder.
>>> Executed the REST request again:
>>>
>>> curl -i -k -u admin:admin-password -X GET
>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>
>>> and I expected to see some logs, e.g. from the "doGet()"
>>> method (I'm not sure about the "@Configure" method, when
>>> should a method with that annotation be executed?), but it
>>> seems it does not see the class.
>>> So I tried to configure my topology to use the dispatch (as
>>> written on [1]):
>>>
>>> <service>
>>> <role>WEBHDFS</role>
>>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>>
>>> <dispatch>
>>> <classname>com.test.ext.MyDispatch</classname>
>>> <use-two-way-ssl>false</use-two-way-ssl>
>>> </dispatch>
>>> </service>
>>>
>>> and after saving logs say:
>>>
>>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) -
>>> Gateway processing failed: javax.servlet.ServletException:
>>> org.apache.shiro.subject.ExecutionException:
>>> java.security.PrivilegedActionException:
>>> javax.servlet.ServletException:
>>> javax.servlet.ServletException:
>>> java.lang.ClassNotFoundException: com.test.ext.MyDispatch
>>> javax.servlet.ServletException:
>>> org.apache.shiro.subject.ExecutionException:
>>> java.security.PrivilegedActionException:
>>> javax.servlet.ServletException:
>>> javax.servlet.ServletException:
>>> *java.lang.ClassNotFoundException: com.test.ext.MyDispatch*
>>>
>>> so it's not seeing my class.
>>> What am I missing?
>>>
>>> Thank you!
>>>
>>> Matteo
>>>
>>>
>>> [1]
>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>>
>>>
>>> On 14/03/19 20:47, Sandeep Moré wrote:
>>>> Hello Matteo,
>>>>
>>>> I don't think this is a right way to add filters to Knox,
>>>> if you want to add some custom logic between your Knox and
>>>> the backend you can write custom dispatch [1]
>>>> you can add this new jar in the ext folder and it should be
>>>> picked up by Knox on start-up.
>>>>
>>>> [1]
>>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>>
>>>>
>>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
>>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to add a filter on a Knox instance so that
>>>> I'll be able to add a custom logic to (every /
>>>> specific) REST requests to Knox.
>>>> I thought to use the "Class Path" feature [1], so I
>>>> created a Maven Java project, generated a ".jar" file
>>>> and placed it in the "$GATEWAY_HOME/ext" folder
>>>> Then, I thought it was necessary to link my custom
>>>> class [2] on the "gateway.xml" files of the resources I
>>>> wanted, e.g.
>>>>
>>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>>
>>>> added:
>>>>
>>>> <filter>
>>>> <role>rewrite</role>
>>>> <name>url-rewrite</name>
>>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>>> </filter>
>>>>
>>>> then I made a REST request to Knox:
>>>>
>>>> curl -i -k -u admin:admin-password -X GET
>>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>>
>>>> but my filter is not called at all (I cannot see the
>>>> log) and I'm not sure whether it's because the class is
>>>> not loaded or the filter is placed in the wrong place
>>>> or whatever.
>>>>
>>>> On [4] you can see the project I have built.
>>>>
>>>> So my questions are:
>>>>
>>>> * what am I doing wrong or missing?
>>>> * is there a better way to do that? I just need to
>>>> add a logic when executing REST requests to Knox
>>>> and make another REST call to an external service I
>>>> need.
>>>>
>>>>
>>>> Thank you!
>>>> Regards,
>>>> Matteo
>>>>
>>>> [1]
>>>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>>> [2]
>>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>>> [3]
>>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>>
>>>>
>>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Thanks!
I checked that too, my ".jar" has the same permissions of the other
files in "lib/" folder (e.g. "gateway-shell-1.2.0.jar").
I see your point about the script [1], but I also see that the original
gateway.sh (for Knox v1.2.0) [2] is not doing anything special for the
"ext/" folder, or am I wrong?
Regards,
Matteo
[1]
https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
[2]
https://github.com/apache/knox/blob/v1.2.0/gateway-release/home/bin/gateway.sh
On 15/03/19 15:27, Sandeep Moré wrote:
> Ah, i missed the launcher.cfg, I see,
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh does
> not take into account a lot of environmental variables and new stuff
> that is included in the
> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
> that script is due for an update.
>
> The only other thing I can think of are the file permissions, make
> sure that your file can be read by knox.
>
> Best,
> Sandeep
>
>
>
> On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni
> <skylark17@apache.org <ma...@apache.org>> wrote:
>
> Hi,
>
> I tried also the "lib/" folder, but same problem.
> About the configuration for classpath yes it should be correct
> because I have the "bin/gateway.cfg" file containing:
>
> #Created from
> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
> #Thu Mar 14 10:10:31 GMT 2019
> GATEWAY_HOME=${launcher.dir}/..
> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name
> <http://launcher.name>}-log4j.properties
> main.class=org.apache.knox.gateway.GatewayServer
> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>
> so it should load the ".jar" files in those folders.
>
> Is there anything I can do to debug this?
> FYI I'm using this Docker image [1] to run Knox and after adding
> files to "ext/" or "lib/" I use this script [2] to restart Knox
> and it seems to restart well, but still not seeing my classes in
> classpath.
>
> Thanks
>
>
> [1] https://github.com/moresandeep/knox-dev-docker
> [2]
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>
>
> On 15/03/19 14:56, Sandeep Moré wrote:
>> It should have picked it up, going through the gateway.sh file, I
>> don't see any place where ext folder is added to class path, can
>> you open a BUG for this ?
>>
>> as a workaround for this you can copy your jar into the lib
>> folder and Knox should pick it up on the startup.
>>
>> Hopefully, this should help !
>>
>> Best,
>> Sandepe
>>
>>
>> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>
>> Hi Sandeep,
>>
>> thank you for your answer!
>> Ok so I tried to change my project and adding a simple class
>> like this:
>>
>> package com.test.ext;
>>
>> import java.io.IOException;
>> import java.net.URI;
>> import java.net.URISyntaxException;
>> import javax.servlet.http.HttpServletRequest;
>> import javax.servlet.http.HttpServletResponse;
>> import org.apache.knox.gateway.config.Configure;
>> import org.apache.knox.gateway.config.Default;
>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>> import org.slf4j.Logger;
>> import org.slf4j.LoggerFactory;
>>
>> public class MyDispatch extends AbstractGatewayDispatch {
>>
>> private static final Logger LOG =
>> LoggerFactory.getLogger(MyDispatch.class);
>>
>> @Override
>> public void destroy() {
>> LOG.debug("******* destroy()");
>> }
>>
>> @Configure
>> protected void customMethod(@Default("Test") String test) {
>> LOG.debug("******* @Configure customMethod(): {}", test);
>> }
>>
>> @Override
>> public void doGet(URI url, HttpServletRequest request,
>> HttpServletResponse response)
>> throws IOException, URISyntaxException {
>>
>> LOG.debug("******* doGet() request: {}, {}",
>> request.getMethod(), new URI(request.getRequestURI()));
>>
>> super.doGet(url, request, response);
>> }
>>
>> }
>>
>> made the ".jar" and put in the "ext" folder.
>> Executed the REST request again:
>>
>> curl -i -k -u admin:admin-password -X GET
>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>
>> and I expected to see some logs, e.g. from the "doGet()"
>> method (I'm not sure about the "@Configure" method, when
>> should a method with that annotation be executed?), but it
>> seems it does not see the class.
>> So I tried to configure my topology to use the dispatch (as
>> written on [1]):
>>
>> <service>
>> <role>WEBHDFS</role>
>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>
>> <dispatch>
>> <classname>com.test.ext.MyDispatch</classname>
>> <use-two-way-ssl>false</use-two-way-ssl>
>> </dispatch>
>> </service>
>>
>> and after saving logs say:
>>
>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) -
>> Gateway processing failed: javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException:
>> javax.servlet.ServletException:
>> javax.servlet.ServletException:
>> java.lang.ClassNotFoundException: com.test.ext.MyDispatch
>> javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException:
>> javax.servlet.ServletException:
>> javax.servlet.ServletException:
>> *java.lang.ClassNotFoundException: com.test.ext.MyDispatch*
>>
>> so it's not seeing my class.
>> What am I missing?
>>
>> Thank you!
>>
>> Matteo
>>
>>
>> [1]
>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>
>>
>> On 14/03/19 20:47, Sandeep Moré wrote:
>>> Hello Matteo,
>>>
>>> I don't think this is a right way to add filters to Knox, if
>>> you want to add some custom logic between your Knox and the
>>> backend you can write custom dispatch [1]
>>> you can add this new jar in the ext folder and it should be
>>> picked up by Knox on start-up.
>>>
>>> [1]
>>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>>
>>>
>>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
>>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to add a filter on a Knox instance so that
>>> I'll be able to add a custom logic to (every / specific)
>>> REST requests to Knox.
>>> I thought to use the "Class Path" feature [1], so I
>>> created a Maven Java project, generated a ".jar" file
>>> and placed it in the "$GATEWAY_HOME/ext" folder
>>> Then, I thought it was necessary to link my custom class
>>> [2] on the "gateway.xml" files of the resources I
>>> wanted, e.g.
>>>
>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>
>>> added:
>>>
>>> <filter>
>>> <role>rewrite</role>
>>> <name>url-rewrite</name>
>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>> </filter>
>>>
>>> then I made a REST request to Knox:
>>>
>>> curl -i -k -u admin:admin-password -X GET
>>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>
>>> but my filter is not called at all (I cannot see the
>>> log) and I'm not sure whether it's because the class is
>>> not loaded or the filter is placed in the wrong place or
>>> whatever.
>>>
>>> On [4] you can see the project I have built.
>>>
>>> So my questions are:
>>>
>>> * what am I doing wrong or missing?
>>> * is there a better way to do that? I just need to add
>>> a logic when executing REST requests to Knox and
>>> make another REST call to an external service I need.
>>>
>>>
>>> Thank you!
>>> Regards,
>>> Matteo
>>>
>>> [1]
>>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>> [2]
>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>> [3]
>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>
>>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Sandeep Moré <mo...@gmail.com>.
Ah, i missed the launcher.cfg, I see,
https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
does
not take into account a lot of environmental variables and new stuff that
is included in the
https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh
that script is due for an update.
The only other thing I can think of are the file permissions, make sure
that your file can be read by knox.
Best,
Sandeep
On Fri, Mar 15, 2019 at 10:09 AM Matteo Alessandroni <sk...@apache.org>
wrote:
> Hi,
>
> I tried also the "lib/" folder, but same problem.
> About the configuration for classpath yes it should be correct because I
> have the "bin/gateway.cfg" file containing:
>
> #Created from
> jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
> #Thu Mar 14 10:10:31 GMT 2019
> GATEWAY_HOME=${launcher.dir}/..
> log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name}-log4j.properties
> main.class=org.apache.knox.gateway.GatewayServer
> class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
>
> so it should load the ".jar" files in those folders.
>
> Is there anything I can do to debug this?
> FYI I'm using this Docker image [1] to run Knox and after adding files to
> "ext/" or "lib/" I use this script [2] to restart Knox and it seems to
> restart well, but still not seeing my classes in classpath.
>
> Thanks
>
>
> [1] https://github.com/moresandeep/knox-dev-docker
> [2]
> https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
>
>
> On 15/03/19 14:56, Sandeep Moré wrote:
>
> It should have picked it up, going through the gateway.sh file, I don't
> see any place where ext folder is added to class path, can you open a BUG
> for this ?
>
> as a workaround for this you can copy your jar into the lib folder and
> Knox should pick it up on the startup.
>
> Hopefully, this should help !
>
> Best,
> Sandepe
>
>
> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni <sk...@apache.org>
> wrote:
>
>> Hi Sandeep,
>>
>> thank you for your answer!
>> Ok so I tried to change my project and adding a simple class like this:
>>
>> package com.test.ext;
>>
>> import java.io.IOException;
>> import java.net.URI;
>> import java.net.URISyntaxException;
>> import javax.servlet.http.HttpServletRequest;
>> import javax.servlet.http.HttpServletResponse;
>> import org.apache.knox.gateway.config.Configure;
>> import org.apache.knox.gateway.config.Default;
>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>> import org.slf4j.Logger;
>> import org.slf4j.LoggerFactory;
>>
>> public class MyDispatch extends AbstractGatewayDispatch {
>>
>> private static final Logger LOG =
>> LoggerFactory.getLogger(MyDispatch.class);
>>
>> @Override
>> public void destroy() {
>> LOG.debug("******* destroy()");
>> }
>>
>> @Configure
>> protected void customMethod(@Default("Test") String test) {
>> LOG.debug("******* @Configure customMethod(): {}", test);
>> }
>>
>> @Override
>> public void doGet(URI url, HttpServletRequest request,
>> HttpServletResponse response)
>> throws IOException, URISyntaxException {
>>
>> LOG.debug("******* doGet() request: {}, {}", request.getMethod(),
>> new URI(request.getRequestURI()));
>>
>> super.doGet(url, request, response);
>> }
>>
>> }
>>
>> made the ".jar" and put in the "ext" folder.
>> Executed the REST request again:
>>
>> curl -i -k -u admin:admin-password -X GET '
>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>
>> and I expected to see some logs, e.g. from the "doGet()" method (I'm not
>> sure about the "@Configure" method, when should a method with that
>> annotation be executed?), but it seems it does not see the class.
>> So I tried to configure my topology to use the dispatch (as written on
>> [1]):
>>
>> <service>
>> <role>WEBHDFS</role>
>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>
>> <dispatch>
>> <classname>com.test.ext.MyDispatch</classname>
>> <use-two-way-ssl>false</use-two-way-ssl>
>> </dispatch>
>> </service>
>>
>> and after saving logs say:
>>
>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
>> processing failed: javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>> javax.servlet.ServletException: java.lang.ClassNotFoundException:
>> com.test.ext.MyDispatch
>> javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>> javax.servlet.ServletException: *java.lang.ClassNotFoundException:
>> com.test.ext.MyDispatch*
>>
>> so it's not seeing my class.
>> What am I missing?
>>
>> Thank you!
>>
>> Matteo
>>
>>
>> [1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>
>>
>> On 14/03/19 20:47, Sandeep Moré wrote:
>>
>> Hello Matteo,
>>
>> I don't think this is a right way to add filters to Knox, if you want to
>> add some custom logic between your Knox and the backend you can write
>> custom dispatch [1]
>> you can add this new jar in the ext folder and it should be picked up by
>> Knox on start-up.
>>
>> [1]
>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>
>>
>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <
>> skylark17@apache.org> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to add a filter on a Knox instance so that I'll be able to
>>> add a custom logic to (every / specific) REST requests to Knox.
>>> I thought to use the "Class Path" feature [1], so I created a Maven Java
>>> project, generated a ".jar" file and placed it in the "$GATEWAY_HOME/ext"
>>> folder
>>> Then, I thought it was necessary to link my custom class [2] on the
>>> "gateway.xml" files of the resources I wanted, e.g.
>>>
>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>
>>> added:
>>>
>>> <filter>
>>> <role>rewrite</role>
>>> <name>url-rewrite</name>
>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>> </filter>
>>>
>>> then I made a REST request to Knox:
>>>
>>> curl -i -k -u admin:admin-password -X GET '
>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>
>>> but my filter is not called at all (I cannot see the log) and I'm not
>>> sure whether it's because the class is not loaded or the filter is placed
>>> in the wrong place or whatever.
>>>
>>> On [4] you can see the project I have built.
>>>
>>> So my questions are:
>>>
>>> - what am I doing wrong or missing?
>>> - is there a better way to do that? I just need to add a logic when
>>> executing REST requests to Knox and make another REST call to an external
>>> service I need.
>>>
>>>
>>> Thank you!
>>> Regards,
>>> Matteo
>>>
>>> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>> [2]
>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>> [3]
>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>
>>>
>>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Hi,
I tried also the "lib/" folder, but same problem.
About the configuration for classpath yes it should be correct because I
have the "bin/gateway.cfg" file containing:
#Created from jar:file:/knox-runtime/bin/gateway.jar!/META-INF/launcher.cfg
#Thu Mar 14 10:10:31 GMT 2019
GATEWAY_HOME=${launcher.dir}/..
log4j.configuration=${GATEWAY_HOME}/conf/${launcher.name}-log4j.properties
main.class=org.apache.knox.gateway.GatewayServer
class.path=../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
so it should load the ".jar" files in those folders.
Is there anything I can do to debug this?
FYI I'm using this Docker image [1] to run Knox and after adding files
to "ext/" or "lib/" I use this script [2] to restart Knox and it seems
to restart well, but still not seeing my classes in classpath.
Thanks
[1] https://github.com/moresandeep/knox-dev-docker
[2]
https://github.com/moresandeep/knox-dev-docker/blob/master/build/gateway.sh
On 15/03/19 14:56, Sandeep Moré wrote:
> It should have picked it up, going through the gateway.sh file, I
> don't see any place where ext folder is added to class path, can you
> open a BUG for this ?
>
> as a workaround for this you can copy your jar into the lib folder and
> Knox should pick it up on the startup.
>
> Hopefully, this should help !
>
> Best,
> Sandepe
>
>
> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni
> <skylark17@apache.org <ma...@apache.org>> wrote:
>
> Hi Sandeep,
>
> thank you for your answer!
> Ok so I tried to change my project and adding a simple class like
> this:
>
> package com.test.ext;
>
> import java.io.IOException;
> import java.net.URI;
> import java.net.URISyntaxException;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> import org.apache.knox.gateway.config.Configure;
> import org.apache.knox.gateway.config.Default;
> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> public class MyDispatch extends AbstractGatewayDispatch {
>
> private static final Logger LOG =
> LoggerFactory.getLogger(MyDispatch.class);
>
> @Override
> public void destroy() {
> LOG.debug("******* destroy()");
> }
>
> @Configure
> protected void customMethod(@Default("Test") String test) {
> LOG.debug("******* @Configure customMethod(): {}", test);
> }
>
> @Override
> public void doGet(URI url, HttpServletRequest request,
> HttpServletResponse response)
> throws IOException, URISyntaxException {
>
> LOG.debug("******* doGet() request: {}, {}",
> request.getMethod(), new URI(request.getRequestURI()));
>
> super.doGet(url, request, response);
> }
>
> }
>
> made the ".jar" and put in the "ext" folder.
> Executed the REST request again:
>
> curl -i -k -u admin:admin-password -X GET
> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>
> and I expected to see some logs, e.g. from the "doGet()" method
> (I'm not sure about the "@Configure" method, when should a method
> with that annotation be executed?), but it seems it does not see
> the class.
> So I tried to configure my topology to use the dispatch (as
> written on [1]):
>
> <service>
> <role>WEBHDFS</role>
> <url>http://hadoop-namenode:50070/webhdfs</url>
>
> <dispatch>
> <classname>com.test.ext.MyDispatch</classname>
> <use-two-way-ssl>false</use-two-way-ssl>
> </dispatch>
> </service>
>
> and after saving logs say:
>
> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
> processing failed: javax.servlet.ServletException:
> org.apache.shiro.subject.ExecutionException:
> java.security.PrivilegedActionException:
> javax.servlet.ServletException: javax.servlet.ServletException:
> java.lang.ClassNotFoundException: com.test.ext.MyDispatch
> javax.servlet.ServletException:
> org.apache.shiro.subject.ExecutionException:
> java.security.PrivilegedActionException:
> javax.servlet.ServletException: javax.servlet.ServletException:
> *java.lang.ClassNotFoundException: com.test.ext.MyDispatch*
>
> so it's not seeing my class.
> What am I missing?
>
> Thank you!
>
> Matteo
>
>
> [1]
> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>
>
> On 14/03/19 20:47, Sandeep Moré wrote:
>> Hello Matteo,
>>
>> I don't think this is a right way to add filters to Knox, if you
>> want to add some custom logic between your Knox and the backend
>> you can write custom dispatch [1]
>> you can add this new jar in the ext folder and it should be
>> picked up by Knox on start-up.
>>
>> [1]
>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>
>>
>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
>> <skylark17@apache.org <ma...@apache.org>> wrote:
>>
>> Hi,
>>
>> I'm trying to add a filter on a Knox instance so that I'll be
>> able to add a custom logic to (every / specific) REST
>> requests to Knox.
>> I thought to use the "Class Path" feature [1], so I created a
>> Maven Java project, generated a ".jar" file and placed it in
>> the "$GATEWAY_HOME/ext" folder
>> Then, I thought it was necessary to link my custom class [2]
>> on the "gateway.xml" files of the resources I wanted, e.g.
>>
>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>
>> added:
>>
>> <filter>
>> <role>rewrite</role>
>> <name>url-rewrite</name>
>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>> </filter>
>>
>> then I made a REST request to Knox:
>>
>> curl -i -k -u admin:admin-password -X GET
>> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>
>> but my filter is not called at all (I cannot see the log) and
>> I'm not sure whether it's because the class is not loaded or
>> the filter is placed in the wrong place or whatever.
>>
>> On [4] you can see the project I have built.
>>
>> So my questions are:
>>
>> * what am I doing wrong or missing?
>> * is there a better way to do that? I just need to add a
>> logic when executing REST requests to Knox and make
>> another REST call to an external service I need.
>>
>>
>> Thank you!
>> Regards,
>> Matteo
>>
>> [1]
>> https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>> [2]
>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>> [3]
>> [4] https://github.com/mat-ale/apache-knox-filter
>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Kevin Risden <kr...@apache.org>.
The launcher.cfg should be setting the classpath.
➜ knox git:(master) git grep -F ext/ | grep -F launcher
gateway-adapter/src/main/resources/META-INF/launcher.cfg:class.path =
../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
gateway-demo-ldap-launcher/src/main/resources/META-INF/launcher.cfg:class.path
= ../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
gateway-server-launcher/src/main/resources/META-INF/launcher.cfg:class.path
= ../conf;../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
gateway-shell-launcher/src/main/resources/META-INF/launcher.cfg:class.path
= ../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
knox-cli-launcher/src/main/resources/META-INF/launcher.cfg:class.path =
../lib/*.jar;../dep/*.jar;../ext;../ext/*.jar
Kevin Risden
On Fri, Mar 15, 2019 at 9:56 AM Sandeep Moré <mo...@gmail.com> wrote:
> It should have picked it up, going through the gateway.sh file, I don't
> see any place where ext folder is added to class path, can you open a BUG
> for this ?
>
> as a workaround for this you can copy your jar into the lib folder and
> Knox should pick it up on the startup.
>
> Hopefully, this should help !
>
> Best,
> Sandepe
>
>
> On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni <sk...@apache.org>
> wrote:
>
>> Hi Sandeep,
>>
>> thank you for your answer!
>> Ok so I tried to change my project and adding a simple class like this:
>>
>> package com.test.ext;
>>
>> import java.io.IOException;
>> import java.net.URI;
>> import java.net.URISyntaxException;
>> import javax.servlet.http.HttpServletRequest;
>> import javax.servlet.http.HttpServletResponse;
>> import org.apache.knox.gateway.config.Configure;
>> import org.apache.knox.gateway.config.Default;
>> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
>> import org.slf4j.Logger;
>> import org.slf4j.LoggerFactory;
>>
>> public class MyDispatch extends AbstractGatewayDispatch {
>>
>> private static final Logger LOG =
>> LoggerFactory.getLogger(MyDispatch.class);
>>
>> @Override
>> public void destroy() {
>> LOG.debug("******* destroy()");
>> }
>>
>> @Configure
>> protected void customMethod(@Default("Test") String test) {
>> LOG.debug("******* @Configure customMethod(): {}", test);
>> }
>>
>> @Override
>> public void doGet(URI url, HttpServletRequest request,
>> HttpServletResponse response)
>> throws IOException, URISyntaxException {
>>
>> LOG.debug("******* doGet() request: {}, {}", request.getMethod(),
>> new URI(request.getRequestURI()));
>>
>> super.doGet(url, request, response);
>> }
>>
>> }
>>
>> made the ".jar" and put in the "ext" folder.
>> Executed the REST request again:
>>
>> curl -i -k -u admin:admin-password -X GET '
>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>
>> and I expected to see some logs, e.g. from the "doGet()" method (I'm not
>> sure about the "@Configure" method, when should a method with that
>> annotation be executed?), but it seems it does not see the class.
>> So I tried to configure my topology to use the dispatch (as written on
>> [1]):
>>
>> <service>
>> <role>WEBHDFS</role>
>> <url>http://hadoop-namenode:50070/webhdfs</url>
>>
>> <dispatch>
>> <classname>com.test.ext.MyDispatch</classname>
>> <use-two-way-ssl>false</use-two-way-ssl>
>> </dispatch>
>> </service>
>>
>> and after saving logs say:
>>
>> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
>> processing failed: javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>> javax.servlet.ServletException: java.lang.ClassNotFoundException:
>> com.test.ext.MyDispatch
>> javax.servlet.ServletException:
>> org.apache.shiro.subject.ExecutionException:
>> java.security.PrivilegedActionException: javax.servlet.ServletException:
>> javax.servlet.ServletException: *java.lang.ClassNotFoundException:
>> com.test.ext.MyDispatch*
>>
>> so it's not seeing my class.
>> What am I missing?
>>
>> Thank you!
>>
>> Matteo
>>
>>
>> [1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>>
>>
>> On 14/03/19 20:47, Sandeep Moré wrote:
>>
>> Hello Matteo,
>>
>> I don't think this is a right way to add filters to Knox, if you want to
>> add some custom logic between your Knox and the backend you can write
>> custom dispatch [1]
>> you can add this new jar in the ext folder and it should be picked up by
>> Knox on start-up.
>>
>> [1]
>> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>>
>>
>> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <
>> skylark17@apache.org> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to add a filter on a Knox instance so that I'll be able to
>>> add a custom logic to (every / specific) REST requests to Knox.
>>> I thought to use the "Class Path" feature [1], so I created a Maven Java
>>> project, generated a ".jar" file and placed it in the "$GATEWAY_HOME/ext"
>>> folder
>>> Then, I thought it was necessary to link my custom class [2] on the
>>> "gateway.xml" files of the resources I wanted, e.g.
>>>
>>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>>
>>> added:
>>>
>>> <filter>
>>> <role>rewrite</role>
>>> <name>url-rewrite</name>
>>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>>> </filter>
>>>
>>> then I made a REST request to Knox:
>>>
>>> curl -i -k -u admin:admin-password -X GET '
>>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>>
>>> but my filter is not called at all (I cannot see the log) and I'm not
>>> sure whether it's because the class is not loaded or the filter is placed
>>> in the wrong place or whatever.
>>>
>>> On [4] you can see the project I have built.
>>>
>>> So my questions are:
>>>
>>> - what am I doing wrong or missing?
>>> - is there a better way to do that? I just need to add a logic when
>>> executing REST requests to Knox and make another REST call to an external
>>> service I need.
>>>
>>>
>>> Thank you!
>>> Regards,
>>> Matteo
>>>
>>> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>>> [2]
>>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>>> [3]
>>> [4] https://github.com/mat-ale/apache-knox-filter
>>>
>>>
>>>
>>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Sandeep Moré <mo...@gmail.com>.
It should have picked it up, going through the gateway.sh file, I don't see
any place where ext folder is added to class path, can you open a BUG for
this ?
as a workaround for this you can copy your jar into the lib folder and Knox
should pick it up on the startup.
Hopefully, this should help !
Best,
Sandepe
On Fri, Mar 15, 2019 at 5:37 AM Matteo Alessandroni <sk...@apache.org>
wrote:
> Hi Sandeep,
>
> thank you for your answer!
> Ok so I tried to change my project and adding a simple class like this:
>
> package com.test.ext;
>
> import java.io.IOException;
> import java.net.URI;
> import java.net.URISyntaxException;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> import org.apache.knox.gateway.config.Configure;
> import org.apache.knox.gateway.config.Default;
> import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
>
> public class MyDispatch extends AbstractGatewayDispatch {
>
> private static final Logger LOG =
> LoggerFactory.getLogger(MyDispatch.class);
>
> @Override
> public void destroy() {
> LOG.debug("******* destroy()");
> }
>
> @Configure
> protected void customMethod(@Default("Test") String test) {
> LOG.debug("******* @Configure customMethod(): {}", test);
> }
>
> @Override
> public void doGet(URI url, HttpServletRequest request,
> HttpServletResponse response)
> throws IOException, URISyntaxException {
>
> LOG.debug("******* doGet() request: {}, {}", request.getMethod(),
> new URI(request.getRequestURI()));
>
> super.doGet(url, request, response);
> }
>
> }
>
> made the ".jar" and put in the "ext" folder.
> Executed the REST request again:
>
> curl -i -k -u admin:admin-password -X GET '
> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>
> and I expected to see some logs, e.g. from the "doGet()" method (I'm not
> sure about the "@Configure" method, when should a method with that
> annotation be executed?), but it seems it does not see the class.
> So I tried to configure my topology to use the dispatch (as written on
> [1]):
>
> <service>
> <role>WEBHDFS</role>
> <url>http://hadoop-namenode:50070/webhdfs</url>
>
> <dispatch>
> <classname>com.test.ext.MyDispatch</classname>
> <use-two-way-ssl>false</use-two-way-ssl>
> </dispatch>
> </service>
>
> and after saving logs say:
>
> ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway processing
> failed: javax.servlet.ServletException:
> org.apache.shiro.subject.ExecutionException:
> java.security.PrivilegedActionException: javax.servlet.ServletException:
> javax.servlet.ServletException: java.lang.ClassNotFoundException:
> com.test.ext.MyDispatch
> javax.servlet.ServletException:
> org.apache.shiro.subject.ExecutionException:
> java.security.PrivilegedActionException: javax.servlet.ServletException:
> javax.servlet.ServletException: *java.lang.ClassNotFoundException:
> com.test.ext.MyDispatch*
>
> so it's not seeing my class.
> What am I missing?
>
> Thank you!
>
> Matteo
>
>
> [1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
>
>
> On 14/03/19 20:47, Sandeep Moré wrote:
>
> Hello Matteo,
>
> I don't think this is a right way to add filters to Knox, if you want to
> add some custom logic between your Knox and the backend you can write
> custom dispatch [1]
> you can add this new jar in the ext folder and it should be picked up by
> Knox on start-up.
>
> [1]
> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>
>
> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <sk...@apache.org>
> wrote:
>
>> Hi,
>>
>> I'm trying to add a filter on a Knox instance so that I'll be able to add
>> a custom logic to (every / specific) REST requests to Knox.
>> I thought to use the "Class Path" feature [1], so I created a Maven Java
>> project, generated a ".jar" file and placed it in the "$GATEWAY_HOME/ext"
>> folder
>> Then, I thought it was necessary to link my custom class [2] on the
>> "gateway.xml" files of the resources I wanted, e.g.
>>
>> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>>
>> added:
>>
>> <filter>
>> <role>rewrite</role>
>> <name>url-rewrite</name>
>> <class>com.test.knox.MyUrlRewriteServletFilter</class>
>> </filter>
>>
>> then I made a REST request to Knox:
>>
>> curl -i -k -u admin:admin-password -X GET '
>> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>>
>> but my filter is not called at all (I cannot see the log) and I'm not
>> sure whether it's because the class is not loaded or the filter is placed
>> in the wrong place or whatever.
>>
>> On [4] you can see the project I have built.
>>
>> So my questions are:
>>
>> - what am I doing wrong or missing?
>> - is there a better way to do that? I just need to add a logic when
>> executing REST requests to Knox and make another REST call to an external
>> service I need.
>>
>>
>> Thank you!
>> Regards,
>> Matteo
>>
>> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
>> [2]
>> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
>> [3]
>> [4] https://github.com/mat-ale/apache-knox-filter
>>
>>
>>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Matteo Alessandroni <sk...@apache.org>.
Hi Sandeep,
thank you for your answer!
Ok so I tried to change my project and adding a simple class like this:
package com.test.ext;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.knox.gateway.config.Configure;
import org.apache.knox.gateway.config.Default;
import org.apache.knox.gateway.dispatch.AbstractGatewayDispatch;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class MyDispatch extends AbstractGatewayDispatch {
private static final Logger LOG =
LoggerFactory.getLogger(MyDispatch.class);
@Override
public void destroy() {
LOG.debug("******* destroy()");
}
@Configure
protected void customMethod(@Default("Test") String test) {
LOG.debug("******* @Configure customMethod(): {}", test);
}
@Override
public void doGet(URI url, HttpServletRequest request,
HttpServletResponse response)
throws IOException, URISyntaxException {
LOG.debug("******* doGet() request: {}, {}",
request.getMethod(), new URI(request.getRequestURI()));
super.doGet(url, request, response);
}
}
made the ".jar" and put in the "ext" folder.
Executed the REST request again:
curl -i -k -u admin:admin-password -X GET
'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
and I expected to see some logs, e.g. from the "doGet()" method (I'm not
sure about the "@Configure" method, when should a method with that
annotation be executed?), but it seems it does not see the class.
So I tried to configure my topology to use the dispatch (as written on [1]):
<service>
<role>WEBHDFS</role>
<url>http://hadoop-namenode:50070/webhdfs</url>
<dispatch>
<classname>com.test.ext.MyDispatch</classname>
<use-two-way-ssl>false</use-two-way-ssl>
</dispatch>
</service>
and after saving logs say:
ERROR knox.gateway (GatewayFilter.java:doFilter(170)) - Gateway
processing failed: javax.servlet.ServletException:
org.apache.shiro.subject.ExecutionException:
java.security.PrivilegedActionException: javax.servlet.ServletException:
javax.servlet.ServletException: java.lang.ClassNotFoundException:
com.test.ext.MyDispatch
javax.servlet.ServletException:
org.apache.shiro.subject.ExecutionException:
java.security.PrivilegedActionException: javax.servlet.ServletException:
javax.servlet.ServletException: *java.lang.ClassNotFoundException:
com.test.ext.MyDispatch*
so it's not seeing my class.
What am I missing?
Thank you!
Matteo
[1] https://knox.apache.org/books/knox-1-3-0/dev-guide.html#service.xml
On 14/03/19 20:47, Sandeep Moré wrote:
> Hello Matteo,
>
> I don't think this is a right way to add filters to Knox, if you want
> to add some custom logic between your Knox and the backend you can
> write custom dispatch [1]
> you can add this new jar in the ext folder and it should be picked up
> by Knox on start-up.
>
> [1]
> https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
>
>
> On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni
> <skylark17@apache.org <ma...@apache.org>> wrote:
>
> Hi,
>
> I'm trying to add a filter on a Knox instance so that I'll be able
> to add a custom logic to (every / specific) REST requests to Knox.
> I thought to use the "Class Path" feature [1], so I created a
> Maven Java project, generated a ".jar" file and placed it in the
> "$GATEWAY_HOME/ext" folder
> Then, I thought it was necessary to link my custom class [2] on
> the "gateway.xml" files of the resources I wanted, e.g.
>
> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>
> added:
>
> <filter>
> <role>rewrite</role>
> <name>url-rewrite</name>
> <class>com.test.knox.MyUrlRewriteServletFilter</class>
> </filter>
>
> then I made a REST request to Knox:
>
> curl -i -k -u admin:admin-password -X GET
> 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>
> but my filter is not called at all (I cannot see the log) and I'm
> not sure whether it's because the class is not loaded or the
> filter is placed in the wrong place or whatever.
>
> On [4] you can see the project I have built.
>
> So my questions are:
>
> * what am I doing wrong or missing?
> * is there a better way to do that? I just need to add a logic
> when executing REST requests to Knox and make another REST
> call to an external service I need.
>
>
> Thank you!
> Regards,
> Matteo
>
> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
> [2]
> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
> [3]
> [4] https://github.com/mat-ale/apache-knox-filter
>
>
Re: Add a filter to intercept Knox requests and be able to add a
custom logic
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Matteo,
I don't think this is a right way to add filters to Knox, if you want to
add some custom logic between your Knox and the backend you can write
custom dispatch [1]
you can add this new jar in the ext folder and it should be picked up by
Knox on start-up.
[1]
https://knox.apache.org/books/knox-1-3-0/dev-guide.html#Custom+Dispatch+Dependency+Injection
On Thu, Mar 14, 2019 at 12:52 PM Matteo Alessandroni <sk...@apache.org>
wrote:
> Hi,
>
> I'm trying to add a filter on a Knox instance so that I'll be able to add
> a custom logic to (every / specific) REST requests to Knox.
> I thought to use the "Class Path" feature [1], so I created a Maven Java
> project, generated a ".jar" file and placed it in the "$GATEWAY_HOME/ext"
> folder
> Then, I thought it was necessary to link my custom class [2] on the
> "gateway.xml" files of the resources I wanted, e.g.
>
> ./data/deployments/sandbox.topo.16977ef2478/%2F/WEB-INF/gateway.xml
>
> added:
>
> <filter>
> <role>rewrite</role>
> <name>url-rewrite</name>
> <class>com.test.knox.MyUrlRewriteServletFilter</class>
> </filter>
>
> then I made a REST request to Knox:
>
> curl -i -k -u admin:admin-password -X GET '
> https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
>
> but my filter is not called at all (I cannot see the log) and I'm not sure
> whether it's because the class is not loaded or the filter is placed in the
> wrong place or whatever.
>
> On [4] you can see the project I have built.
>
> So my questions are:
>
> - what am I doing wrong or missing?
> - is there a better way to do that? I just need to add a logic when
> executing REST requests to Knox and make another REST call to an external
> service I need.
>
>
> Thank you!
> Regards,
> Matteo
>
> [1] https://knox.apache.org/books/knox-1-2-0/dev-guide.html#Class+Path
> [2]
> https://github.com/mat-ale/apache-knox-filter/blob/master/src/main/java/com/plainid/ext/MyUrlRewriteServletFilter.java
> [3]
> [4] https://github.com/mat-ale/apache-knox-filter
>
>
>