You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Achim Kraus <ac...@gmx.net> on 2021/04/20 10:02:06 UTC

http core 5.0.2 - server setup for proxy request

Dear apache-hc devlist,

I currently try to update the Eclipse/Californium http2coap cross proxy
to use version 5 of the apache-hc-core library instead of version 4.

I'm struggling to get the http-proxy function working again.

My client sends a http request in proxy manner with the host in the
request-line according wireshark:

--------- wireshark ----------------
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 52604, Dst Port: 8080, Seq: 1,
Ack: 1, Len: 171
Hypertext Transfer Protocol
     GET http://localhost:5685/coap-target/coap: HTTP/1.1\r\n
         [Expert Info (Chat/Sequence): GET
http://localhost:5685/coap-target/coap: HTTP/1.1\r\n]
         Request Method: GET
         Request URI: http://localhost:5685/coap-target/coap:
         Request Version: HTTP/1.1
------------------------------------

I use a setup close to the one in the example in

https://github.com/apache/httpcomponents-core/tree/5.1.x/httpcore5/src/test/java/org/apache/hc/core5/http/examples/AsyncFileServerExample.java

registering handlers with:

.register("http*", new AsyncServerRequestHandler() ... for proxy request)
.register("*", new AsyncServerRequestHandler() ... for default normal
request)

similar as before with version 4.

But with version 5 this seems to not work any longer.
Now all requests are passed to the "*" default normal handler.
Logging the RequestLine(httpRequest) and URI in that handler results in:

[HttpServer]: http-proxy received GET /coap-target/coap: HTTP/1.1
[HttpServer]: URI: http://localhost:5685/coap-target/coap:

So, the URI is as intended, but the RequestLine not. I guess, that this
may also be the root cause for not selecting the "http*" proxy handler.

Is there a way to make this working again?
Or what is the preferred way to register handler for incoming
proxy-request (with the final destination included)?
(I only found an example for an reverse proxy, that uses "*".
But I didn't find some stuff for a forward proxy.)

Best regards
Achim Kraus

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: http core 5.0.2 - server setup for proxy request

Posted by Achim Kraus <ac...@gmx.net>.
Hello Oleg,

Thanks for your help and proposal!

> This can be done by replacing the terminal (main) filter in the request
> execution chain
>
> ```
>
> .replaceFilter(StandardFilter.MAIN_HANDLER.name(), new
> TerminalAsyncServerFilter(new DefaultAsyncResponseExchangeHandlerFactory(
>          new HttpRequestMapper<Supplier<AsyncServerExchangeHandler>>() {
>
>              @Override
>              public Supplier<AsyncServerExchangeHandler>
> resolve(HttpRequest request, HttpContext context) throws HttpException {
>                  // Do custom request mapping
>                  return null;
>              }
>
>          })))
>
> ```

I slightly modified the idea an put a proxy-filter before the default
main-filter.
In case of a proxy request, it's "handled" there, otherwise the chain is
"proceeded". The hard thing is to distinguish between virtual hostnames
and proxy request. I didn't really solve that right now, I just use
"uri.getScheme() != null".

best regards
Achim Kraus

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: http core 5.0.2 - server setup for proxy request

Posted by Oleg Kalnichevski <ol...@apache.org>.
On 4/20/2021 2:33 PM, Achim Kraus wrote:
> Hi Oleg,
>
> thanks for the fast response!
>
> 1. "virtual host": that would be a solution, but generic forwarding
> proxies are usually don't have a list of "destination hosts". And "*"
> seems not working as name of the "virtual host". Maybe an idea for a
> future extension?
>
> 2. "Custom HttpRequestMapper": Using AsyncServerBootstrap seems not
> offering something, where I can set a custom implementation. So, what
> need to be done in order to use such a Custom HttpRequestMapper?
> (The offered setter for a LookupRegistry seem to be one step too far,
> there is already only the path used.)
>
This can be done by replacing the terminal (main) filter in the request 
execution chain

```

.replaceFilter(StandardFilter.MAIN_HANDLER.name(), new 
TerminalAsyncServerFilter(new DefaultAsyncResponseExchangeHandlerFactory(
         new HttpRequestMapper<Supplier<AsyncServerExchangeHandler>>() {

             @Override
             public Supplier<AsyncServerExchangeHandler> 
resolve(HttpRequest request, HttpContext context) throws HttpException {
                 // Do custom request mapping
                 return null;
             }

         })))

```

Please note this will disable all other request mappings generated by 
the same `AsyncServerBootstrap` class


> 3. "or doing extra request routing inside your request handlers":
> Yes, that's what I implemented now. But the old solution, register with
> "http*" is in my opinion far more elegant :-).
>
I am not sure. This mapping mechanism relies on message composition 
specific to HTTP/1.1 protocol version and it works correctly simply by 
accident rather by design.

You are welcome to propose API extensions that would make request 
routing easier or more elegant. One immediate change we could consider 
is making custom request mappers easier to plug in.

Oleg


>
> Am 20.04.21 um 14:14 schrieb Oleg Kalnichevski:
>>
>>
>> On 4/20/2021 12:02 PM, Achim Kraus wrote:
>>> Dear apache-hc devlist,
>>>
>>> I currently try to update the Eclipse/Californium http2coap cross proxy
>>> to use version 5 of the apache-hc-core library instead of version 4.
>>>
>>> I'm struggling to get the http-proxy function working again.
>>>
>>> My client sends a http request in proxy manner with the host in the
>>> request-line according wireshark:
>>>
>>> --------- wireshark ----------------
>>> Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
>>> Transmission Control Protocol, Src Port: 52604, Dst Port: 8080, Seq: 1,
>>> Ack: 1, Len: 171
>>> Hypertext Transfer Protocol
>>>     GET http://localhost:5685/coap-target/coap: HTTP/1.1\r\n
>>>         [Expert Info (Chat/Sequence): GET
>>> http://localhost:5685/coap-target/coap: HTTP/1.1\r\n]
>>>         Request Method: GET
>>>         Request URI: http://localhost:5685/coap-target/coap:
>>>         Request Version: HTTP/1.1
>>> ------------------------------------
>>>
>>> I use a setup close to the one in the example in
>>>
>>> https://github.com/apache/httpcomponents-core/tree/5.1.x/httpcore5/src/test/java/org/apache/hc/core5/http/examples/AsyncFileServerExample.java 
>>>
>>>
>>>
>>> registering handlers with:
>>>
>>> .register("http*", new AsyncServerRequestHandler() ... for proxy 
>>> request)
>>> .register("*", new AsyncServerRequestHandler() ... for default normal
>>> request)
>>>
>>> similar as before with version 4.
>>>
>>> But with version 5 this seems to not work any longer.
>>> Now all requests are passed to the "*" default normal handler.
>>> Logging the RequestLine(httpRequest) and URI in that handler results 
>>> in:
>>>
>>> [HttpServer]: http-proxy received GET /coap-target/coap: HTTP/1.1
>>> [HttpServer]: URI: http://localhost:5685/coap-target/coap:
>>>
>>> So, the URI is as intended, but the RequestLine not. I guess, that this
>>> may also be the root cause for not selecting the "http*" proxy handler.
>>>
>>> Is there a way to make this working again?
>>> Or what is the preferred way to register handler for incoming
>>> proxy-request (with the final destination included)?
>>> (I only found an example for an reverse proxy, that uses "*".
>>> But I didn't find some stuff for a forward proxy.)
>>>
>>> Best regards
>>> Achim Kraus
>>>
>>
>> Hi Achim
>>
>> Please take a look at the handler resolution logic in HC 5
>>
>> https://github.com/apache/httpcomponents-core/blob/5.0.x/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestHandlerRegistry.java#L103 
>>
>>
>>
>> This is a two step process.
>>
>> 1. the registry matches the host component of the request URI authority
>> to a particular virtual host recognized by the server. If no virtual
>> host has been matched and the host component is not 'localhost' or the
>> canonical hostname, the server responds with status 421.
>>
>> 2. the registry matches the path component of the request URI to
>> particular handler associated with a request path expressed by a
>> pattern. If no handler has been matched the server responds with 
>> status 404
>>
>> If your application routing logic is based on host names, you need to
>> register your handlers with the correct virtual host.
>>
>> If your application routing logic is more complex, you may consider
>> building a custom `HttpRequestMapper` implementation or doing extra
>> request routing inside your request handlers.
>>
>> Hope this helps.
>>
>> Oleg
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: http core 5.0.2 - server setup for proxy request

Posted by Achim Kraus <ac...@gmx.net>.
Hi Oleg,

thanks for the fast response!

1. "virtual host": that would be a solution, but generic forwarding
proxies are usually don't have a list of "destination hosts". And "*"
seems not working as name of the "virtual host". Maybe an idea for a
future extension?

2. "Custom HttpRequestMapper": Using AsyncServerBootstrap seems not
offering something, where I can set a custom implementation. So, what
need to be done in order to use such a Custom HttpRequestMapper?
(The offered setter for a LookupRegistry seem to be one step too far,
there is already only the path used.)

3. "or doing extra request routing inside your request handlers":
Yes, that's what I implemented now. But the old solution, register with
"http*" is in my opinion far more elegant :-).

Best regards
Achim Kraus

Am 20.04.21 um 14:14 schrieb Oleg Kalnichevski:
>
>
> On 4/20/2021 12:02 PM, Achim Kraus wrote:
>> Dear apache-hc devlist,
>>
>> I currently try to update the Eclipse/Californium http2coap cross proxy
>> to use version 5 of the apache-hc-core library instead of version 4.
>>
>> I'm struggling to get the http-proxy function working again.
>>
>> My client sends a http request in proxy manner with the host in the
>> request-line according wireshark:
>>
>> --------- wireshark ----------------
>> Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
>> Transmission Control Protocol, Src Port: 52604, Dst Port: 8080, Seq: 1,
>> Ack: 1, Len: 171
>> Hypertext Transfer Protocol
>>     GET http://localhost:5685/coap-target/coap: HTTP/1.1\r\n
>>         [Expert Info (Chat/Sequence): GET
>> http://localhost:5685/coap-target/coap: HTTP/1.1\r\n]
>>         Request Method: GET
>>         Request URI: http://localhost:5685/coap-target/coap:
>>         Request Version: HTTP/1.1
>> ------------------------------------
>>
>> I use a setup close to the one in the example in
>>
>> https://github.com/apache/httpcomponents-core/tree/5.1.x/httpcore5/src/test/java/org/apache/hc/core5/http/examples/AsyncFileServerExample.java
>>
>>
>> registering handlers with:
>>
>> .register("http*", new AsyncServerRequestHandler() ... for proxy request)
>> .register("*", new AsyncServerRequestHandler() ... for default normal
>> request)
>>
>> similar as before with version 4.
>>
>> But with version 5 this seems to not work any longer.
>> Now all requests are passed to the "*" default normal handler.
>> Logging the RequestLine(httpRequest) and URI in that handler results in:
>>
>> [HttpServer]: http-proxy received GET /coap-target/coap: HTTP/1.1
>> [HttpServer]: URI: http://localhost:5685/coap-target/coap:
>>
>> So, the URI is as intended, but the RequestLine not. I guess, that this
>> may also be the root cause for not selecting the "http*" proxy handler.
>>
>> Is there a way to make this working again?
>> Or what is the preferred way to register handler for incoming
>> proxy-request (with the final destination included)?
>> (I only found an example for an reverse proxy, that uses "*".
>> But I didn't find some stuff for a forward proxy.)
>>
>> Best regards
>> Achim Kraus
>>
>
> Hi Achim
>
> Please take a look at the handler resolution logic in HC 5
>
> https://github.com/apache/httpcomponents-core/blob/5.0.x/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestHandlerRegistry.java#L103
>
>
> This is a two step process.
>
> 1. the registry matches the host component of the request URI authority
> to a particular virtual host recognized by the server. If no virtual
> host has been matched and the host component is not 'localhost' or the
> canonical hostname, the server responds with status 421.
>
> 2. the registry matches the path component of the request URI to
> particular handler associated with a request path expressed by a
> pattern. If no handler has been matched the server responds with status 404
>
> If your application routing logic is based on host names, you need to
> register your handlers with the correct virtual host.
>
> If your application routing logic is more complex, you may consider
> building a custom `HttpRequestMapper` implementation or doing extra
> request routing inside your request handlers.
>
> Hope this helps.
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: http core 5.0.2 - server setup for proxy request

Posted by Oleg Kalnichevski <ol...@apache.org>.

On 4/20/2021 12:02 PM, Achim Kraus wrote:
> Dear apache-hc devlist,
> 
> I currently try to update the Eclipse/Californium http2coap cross proxy
> to use version 5 of the apache-hc-core library instead of version 4.
> 
> I'm struggling to get the http-proxy function working again.
> 
> My client sends a http request in proxy manner with the host in the
> request-line according wireshark:
> 
> --------- wireshark ----------------
> Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
> Transmission Control Protocol, Src Port: 52604, Dst Port: 8080, Seq: 1,
> Ack: 1, Len: 171
> Hypertext Transfer Protocol
>     GET http://localhost:5685/coap-target/coap: HTTP/1.1\r\n
>         [Expert Info (Chat/Sequence): GET
> http://localhost:5685/coap-target/coap: HTTP/1.1\r\n]
>         Request Method: GET
>         Request URI: http://localhost:5685/coap-target/coap:
>         Request Version: HTTP/1.1
> ------------------------------------
> 
> I use a setup close to the one in the example in
> 
> https://github.com/apache/httpcomponents-core/tree/5.1.x/httpcore5/src/test/java/org/apache/hc/core5/http/examples/AsyncFileServerExample.java 
> 
> 
> registering handlers with:
> 
> .register("http*", new AsyncServerRequestHandler() ... for proxy request)
> .register("*", new AsyncServerRequestHandler() ... for default normal
> request)
> 
> similar as before with version 4.
> 
> But with version 5 this seems to not work any longer.
> Now all requests are passed to the "*" default normal handler.
> Logging the RequestLine(httpRequest) and URI in that handler results in:
> 
> [HttpServer]: http-proxy received GET /coap-target/coap: HTTP/1.1
> [HttpServer]: URI: http://localhost:5685/coap-target/coap:
> 
> So, the URI is as intended, but the RequestLine not. I guess, that this
> may also be the root cause for not selecting the "http*" proxy handler.
> 
> Is there a way to make this working again?
> Or what is the preferred way to register handler for incoming
> proxy-request (with the final destination included)?
> (I only found an example for an reverse proxy, that uses "*".
> But I didn't find some stuff for a forward proxy.)
> 
> Best regards
> Achim Kraus
> 

Hi Achim

Please take a look at the handler resolution logic in HC 5

https://github.com/apache/httpcomponents-core/blob/5.0.x/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestHandlerRegistry.java#L103

This is a two step process.

1. the registry matches the host component of the request URI authority 
to a particular virtual host recognized by the server. If no virtual 
host has been matched and the host component is not 'localhost' or the 
canonical hostname, the server responds with status 421.

2. the registry matches the path component of the request URI to 
particular handler associated with a request path expressed by a 
pattern. If no handler has been matched the server responds with status 404

If your application routing logic is based on host names, you need to 
register your handlers with the correct virtual host.

If your application routing logic is more complex, you may consider 
building a custom `HttpRequestMapper` implementation or doing extra 
request routing inside your request handlers.

Hope this helps.

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org