You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Dhruv Goyal <77...@gmail.com> on 2018/07/24 09:14:15 UTC
Issue in writing a rule containing %.
Hi,
I am trying to integrate Knox with a custom service.
Knox version- v0.12
I have many rest calls which contain % as the special characters and I want to integrate these calls with Knox.
Examples:
1./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
2. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Timer.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
3./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.%.CacheMisses&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
4. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=dfs.NNTopUserOpCounts.windowMs=300000.op=__%.user=%&hostname=%&appId=namenode&instanceId=&startTime=1532406804&endTime=1532417604
There other similar calls, here in all these calls we have `%` character in query parameter, we are trying to use a single rule:
<rule dir="IN" name="GRAFANAUI/grafana/inbound/timerq" pattern="*://*:*/**/grafana/*/*/*/*/*/*/timeline/metrics?{metricNames=*%*}?{**}">
<rewrite template="{$serviceUrl[GRAFANAUI]}/api/datasources/proxy/1/ws/v1/timeline/metrics?{metricNames=*%2525*}?{**}"/>
</rule>
In the rewrite template we are trying to replace % with %2525, this is required since we are facing some url decoding issues.
So we have the query parameter "metricNames” with pattern {metricNames=*%*} which works, but in the rewrite template we are using {metricNames=*%2525*} But this wildcard isnt working, is there some other wildcard which can be used instead of ‘*’ in the rewrite template.
Or is there any other way in which we can avoid url decoder issue.
Exception observed:
2018-07-23 06:47:39,276 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false>, direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
2018-07-23 06:47:39,277 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
at java.net.URI.create(URI.java:852)
at org.apache.hadoop.gateway.dispatch.AbstractGatewayDispatch.getDispatchUrl(AbstractGatewayDispatch.java:88)
at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:132)
at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:115)
at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
Thanks & Regards
Dhruv Goyal
Re: Issue in writing a rule containing %.
Posted by Dhruv Goyal <77...@gmail.com>.
We are using HDP-2.6.5 which comes with Knox Version v0.12, Also the HDP’s Knox packaging is a little different.
> On 24-Jul-2018, at 7:57 PM, Dhruv Goyal <77...@gmail.com> wrote:
>
> Hi,
> Thanks Sandeep,
>
> I tried using the same dispatch but still there was no luck I got this exception:
>
> 2018-07-24 14:23:51,128 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532438630&endTime=1532442230&topN=5&topNFunction=max&isBottomN=false <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532438630&endTime=1532442230&topN=5&topNFunction=max&isBottomN=false>, direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532438630&endTime=1532442230&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532438630&endTime=1532442230&topNFunction=max&topN=5&isBottomN=false>
> 2018-07-24 14:23:51,129 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "._"
> java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "._"
> at java.net.URLDecoder.decode(URLDecoder.java:194)
> at org.apache.hadoop.gateway.dispatch.PassAllHeadersNoEncodingDispatch.getDispatchUrl(PassAllHeadersNoEncodingDispatch.java:39)
>
> Is this because we are using Knox version v0.12 ?
>
> Regards
> Dhruv
>
>> On 24-Jul-2018, at 6:48 PM, Sandeep Moré <moresandeep@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Dhruv,
>>
>> In your service definition you might want to use the PassAllHeadersNoEncodingDispatch <https://github.com/apache/knox/blob/90d4339973870fbaa9c48eeae6896fefec95035d/gateway-adapter/src/main/java/org/apache/hadoop/gateway/dispatch/PassAllHeadersNoEncodingDispatch.java> dispatch.
>> We have a Unit Test <https://github.com/apache/knox/blob/64a762f531e87923d3b4ff10141b69ee78904b17/gateway-spi/src/test/java/org/apache/knox/gateway/dispatch/PassAllHeadersNoEncodingDispatchTest.java#L67> for this case in case you want to test it out.
>>
>> Handling this in rewrite rules would be tricky, the dispatch route is pretty simple and should work.
>>
>> Best,
>> Sandeep
>>
>> On Tue, Jul 24, 2018 at 5:14 AM Dhruv Goyal <777.dhruv@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> I am trying to integrate Knox with a custom service.
>> Knox version- v0.12
>>
>> I have many rest calls which contain % as the special characters and I want to integrate these calls with Knox.
>>
>> Examples:
>> 1./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>>
>> 2. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Timer.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>>
>> 3./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.%.CacheMisses&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>>
>> 4. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=dfs.NNTopUserOpCounts.windowMs=300000.op=__%.user=%&hostname=%&appId=namenode&instanceId=&startTime=1532406804&endTime=1532417604
>>
>> There other similar calls, here in all these calls we have `%` character in query parameter, we are trying to use a single rule:
>>
>> <rule dir="IN" name="GRAFANAUI/grafana/inbound/timerq" pattern="*://*:*/**/grafana/*/*/*/*/*/*/timeline/metrics?{metricNames=*%*}?{**}">
>> <rewrite template="{$serviceUrl[GRAFANAUI]}/api/datasources/proxy/1/ws/v1/timeline/metrics?{metricNames=*%2525*}?{**}"/>
>> </rule>
>>
>> In the rewrite template we are trying to replace % with %2525, this is required since we are facing some url decoding issues.
>>
>> So we have the query parameter "metricNames” with pattern {metricNames=*%*} which works, but in the rewrite template we are using {metricNames=*%2525*} But this wildcard isnt working, is there some other wildcard which can be used instead of ‘*’ in the rewrite template.
>> Or is there any other way in which we can avoid url decoder issue.
>>
>>
>> Exception observed:
>>
>>
>> 2018-07-23 06:47:39,276 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false>, direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
>> 2018-07-23 06:47:39,277 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
>> java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
>> at java.net.URI.create(URI.java:852)
>> at org.apache.hadoop.gateway.dispatch.AbstractGatewayDispatch.getDispatchUrl(AbstractGatewayDispatch.java:88)
>> at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:132)
>> at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:115)
>> at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
>> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
>> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
>>
>>
>>
>>
>> Thanks & Regards
>> Dhruv Goyal
>
Re: Issue in writing a rule containing %.
Posted by Dhruv Goyal <77...@gmail.com>.
Hi,
Thanks Sandeep,
I tried using the same dispatch but still there was no luck I got this exception:
2018-07-24 14:23:51,128 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532438630&endTime=1532442230&topN=5&topNFunction=max&isBottomN=false, direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532438630&endTime=1532442230&topNFunction=max&topN=5&isBottomN=false
2018-07-24 14:23:51,129 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "._"
java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "._"
at java.net.URLDecoder.decode(URLDecoder.java:194)
at org.apache.hadoop.gateway.dispatch.PassAllHeadersNoEncodingDispatch.getDispatchUrl(PassAllHeadersNoEncodingDispatch.java:39)
Is this because we are using Knox version v0.12 ?
Regards
Dhruv
> On 24-Jul-2018, at 6:48 PM, Sandeep Moré <mo...@gmail.com> wrote:
>
> Hello Dhruv,
>
> In your service definition you might want to use the PassAllHeadersNoEncodingDispatch <https://github.com/apache/knox/blob/90d4339973870fbaa9c48eeae6896fefec95035d/gateway-adapter/src/main/java/org/apache/hadoop/gateway/dispatch/PassAllHeadersNoEncodingDispatch.java> dispatch.
> We have a Unit Test <https://github.com/apache/knox/blob/64a762f531e87923d3b4ff10141b69ee78904b17/gateway-spi/src/test/java/org/apache/knox/gateway/dispatch/PassAllHeadersNoEncodingDispatchTest.java#L67> for this case in case you want to test it out.
>
> Handling this in rewrite rules would be tricky, the dispatch route is pretty simple and should work.
>
> Best,
> Sandeep
>
> On Tue, Jul 24, 2018 at 5:14 AM Dhruv Goyal <777.dhruv@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I am trying to integrate Knox with a custom service.
> Knox version- v0.12
>
> I have many rest calls which contain % as the special characters and I want to integrate these calls with Knox.
>
> Examples:
> 1./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 2. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Timer.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 3./api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.%.CacheMisses&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 4. /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=dfs.NNTopUserOpCounts.windowMs=300000.op=__%.user=%&hostname=%&appId=namenode&instanceId=&startTime=1532406804&endTime=1532417604
>
> There other similar calls, here in all these calls we have `%` character in query parameter, we are trying to use a single rule:
>
> <rule dir="IN" name="GRAFANAUI/grafana/inbound/timerq" pattern="*://*:*/**/grafana/*/*/*/*/*/*/timeline/metrics?{metricNames=*%*}?{**}">
> <rewrite template="{$serviceUrl[GRAFANAUI]}/api/datasources/proxy/1/ws/v1/timeline/metrics?{metricNames=*%2525*}?{**}"/>
> </rule>
>
> In the rewrite template we are trying to replace % with %2525, this is required since we are facing some url decoding issues.
>
> So we have the query parameter "metricNames” with pattern {metricNames=*%*} which works, but in the rewrite template we are using {metricNames=*%2525*} But this wildcard isnt working, is there some other wildcard which can be used instead of ‘*’ in the rewrite template.
> Or is there any other way in which we can avoid url decoder issue.
>
>
> Exception observed:
>
>
> 2018-07-23 06:47:39,276 DEBUG hadoop.gateway (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false <https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false>, direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
> 2018-07-23 06:47:39,277 ERROR hadoop.gateway (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
> java.lang.IllegalArgumentException: Malformed escape pair at index 118: http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false <http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false>
> at java.net.URI.create(URI.java:852)
> at org.apache.hadoop.gateway.dispatch.AbstractGatewayDispatch.getDispatchUrl(AbstractGatewayDispatch.java:88)
> at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:132)
> at org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:115)
> at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
>
>
>
>
> Thanks & Regards
> Dhruv Goyal
Re: Issue in writing a rule containing %.
Posted by Sandeep Moré <mo...@gmail.com>.
Hello Dhruv,
In your service definition you might want to use the
PassAllHeadersNoEncodingDispatch
<https://github.com/apache/knox/blob/90d4339973870fbaa9c48eeae6896fefec95035d/gateway-adapter/src/main/java/org/apache/hadoop/gateway/dispatch/PassAllHeadersNoEncodingDispatch.java>
dispatch.
We have a Unit Test
<https://github.com/apache/knox/blob/64a762f531e87923d3b4ff10141b69ee78904b17/gateway-spi/src/test/java/org/apache/knox/gateway/dispatch/PassAllHeadersNoEncodingDispatchTest.java#L67>
for this case in case you want to test it out.
Handling this in rewrite rules would be tricky, the dispatch route is
pretty simple and should work.
Best,
Sandeep
On Tue, Jul 24, 2018 at 5:14 AM Dhruv Goyal <77...@gmail.com> wrote:
> Hi,
>
> I am trying to integrate Knox with a custom service.
> Knox version- v0.12
>
> I have many rest calls which contain % as the special characters and I
> want to integrate these calls with Knox.
>
> Examples:
> 1.
> /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 2.
> /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Timer.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 3.
> /api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.%.CacheMisses&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532413786&endTime=1532417386&topN=5&topNFunction=max&isBottomN=false
>
> 4. /
> api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=dfs.NNTopUserOpCounts.windowMs=300000.op=__%.user=%&hostname=%&appId=namenode&instanceId=&startTime=1532406804&endTime=1532417604
>
> There other similar calls, here in all these calls we have `%` character
> in query parameter, we are trying to use a single rule:
>
> <rule dir="IN" name="GRAFANAUI/grafana/inbound/timerq" pattern=
> "*://*:*/**/grafana/*/*/*/*/*/*/timeline/metrics?{metricNames=*%*}?{**}">
> <rewrite template=
> "{$serviceUrl[GRAFANAUI]}/api/datasources/proxy/1/ws/v1/timeline/metrics?{metricNames=*%2525*}?{**}"
> />
> </rule>
>
> In the rewrite template we are trying to replace % with %2525, this is
> required since we are facing some url decoding issues.
>
> So we have the query parameter "metricNames” with pattern
> {metricNames=*%*} which works, but in the rewrite template we are using
> {metricNames=*%2525*} But this wildcard isnt working, is there some
> other wildcard which can be used instead of ‘*’ in the rewrite template.
> Or is there any other way in which we can avoid url decoder issue.
>
>
> Exception observed:
>
>
> 2018-07-23 06:47:39,276 DEBUG hadoop.gateway
> (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL:
> https://192.168.134.119:8443/gateway/default/grafana/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&appId=ambari_server&instanceId=&startTime=1532321602&endTime=1532325202&topN=5&topNFunction=max&isBottomN=false,
> direction: IN via explicit rule: GRAFANAUI/grafana/inbound/query to URL:
> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%25._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false
> 2018-07-23 06:47:39,277 ERROR hadoop.gateway
> (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter:
> java.lang.IllegalArgumentException: Malformed escape pair at index 118:
> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false
> java.lang.IllegalArgumentException: Malformed escape pair at index 118:
> http://platacc003-mst-01.gvs.ggn:3000/api/datasources/proxy/1/ws/v1/timeline/metrics?metricNames=Counter.ReadAllQuery.%._rate&hostname=platacc003-mst-01.gvs.ggn&instanceId=&appId=ambari_server&startTime=1532321602&endTime=1532325202&topNFunction=max&topN=5&isBottomN=false
> at java.net.URI.create(URI.java:852)
> at
> org.apache.hadoop.gateway.dispatch.AbstractGatewayDispatch.getDispatchUrl(AbstractGatewayDispatch.java:88)
> at
> org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:132)
> at
> org.apache.hadoop.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:115)
> at
> org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
> at
> org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
> at
> org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
>
>
>
>
>
> Thanks & Regards
> Dhruv Goyal
>