You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "pjfanning (via GitHub)" <gi...@apache.org> on 2024/02/14 11:13:04 UTC
[I] support making HTTP client requests using HTTP/2 [incubator-pekko-http]
pjfanning opened a new issue, #484:
URL: https://github.com/apache/incubator-pekko-http/issues/484
See https://github.com/matsluni/aws-spi-akka-http/pull/226#issuecomment-1943538007 for context.
It appears that `Http().singleRequest(...)` only supports sending HTTP/1.1 requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "samueleresca (via GitHub)" <gi...@apache.org>.
samueleresca commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2018926929
I was taking a look at this. `Http().singleRequest(...)` a.k.a [Request-Level Client-Side API](https://pekko.apache.org/docs/pekko-http/current/client-side/request-level.html) uses a connection pool (see: [PoolInterface](https://github.com/apache/pekko-http/blob/main/http-core/src/main/scala/org/apache/pekko/http/impl/engine/client/PoolInterface.scala#L49)) to reuse connections. In order to expose an single request Http2 API(i.e.:`Http2().singleRequest(HttpRequest(uri = "http://localhost"))`) and keeping the same behavior of `Http().singleRequest` I think we can proceed as follow:
- Create an `Http2` extension similar to the [Http extension](https://github.com/apache/pekko-http/blob/b6263de8c5582e096bfc5cdd969576d8de438237/http-core/src/main/scala/org/apache/pekko/http/scaladsl/Http.scala#L68) with the `** CLIENT **` and `** CONNECTION POOL **` sections.
- Update the the the [PoolInterface connection flow](https://github.com/apache/pekko-http/blob/main/http-core/src/main/scala/org/apache/pekko/http/impl/engine/client/PoolInterface.scala#L76) to handle both `Http` and `Http2`
Is my understanding / approach right here? Tagging explicitly @raboof as I saw his name in some of the files involved in the changes + some HTTP/2 related work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "samueleresca (via GitHub)" <gi...@apache.org>.
samueleresca commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2068066296
@jrudolph Thanks for the steering.
> I don't think we should do it like this. Especially, we don't want another entry point like Http2(). If we want to allow HTTP2 to be integrated more deeply, it should be done transparently using the existing APIs.
ACK on the above. So the apporach you are suggesting is to skip the initialization of the `NewHostConnectionPool` in case http2 is specified and to use directly the flow coming from `managedPersistentHttp2` for constructing the `PoolInterface`?
> Which is not quite the case right now. NewHostConnectionPool is Flow[RequestContext, ResponseContext] while managedPersistentHttp2 is Flow[HttpRequest, HttpResponse]. That's probably not a blocker, it's more an artifact of the HTTP/1.1 low-level impl not (yet) supporting the more general RequestResponseAssociation protocol to associated requests with responses that HTTP/2 supports.
What is your recommendation here? I'm seeing that `[Request|Response]Context` is a case class wrapping a `Http[Request|Response]`. Should I create a new `PoolInterfaceStage` implementation that handle `Flow[HttpRequest, HttpResponse]` instead of `Flow[RequestContext, ResponseContext]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "jrudolph (via GitHub)" <gi...@apache.org>.
jrudolph commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2056413441
> I think we can proceed as follow
I don't think we should do it like this. Especially, we don't want another entry point like `Http2()`. If we want to allow HTTP2 to be integrated more deeply, it should be done transparently using the existing APIs.
In theory, all existing APIs can be supported by "just" allowing `PoolInterface` to deal with HTTP/2. This might not be a trivial task because of assumptions between the `PoolInterfaceStage` and `NewHostConnectionPool`. Let's say, we ignore automatic negotiation for now and require that you need to configure whether you want to connect to a host via HTTP/1.1 or HTTP/2 (i.e. a new setting in host-connection-pool`).
What needs to be done when HTTP/2 is selected is to replace the code at `PoolInterface.apply`, to create an alternative connection flow that uses a single managed persistent HTTP/2 connection instead of the pool of HTTP/1.1 connections.
https://github.com/apache/pekko-http/blob/87f55da5817b20be224b96d87e984bdfbf883f79/http-core/src/main/scala/org/apache/pekko/http/impl/engine/client/PoolInterface.scala#L76-L79
On the surface, it might be enough to switch out those two lines with the HTTP/2 equivalent. I wouldn't surprised if there are some subtle issues/bugs remaining in `managedPersistentHttp2` which was a late addition to HTTP/2 support in akka-http.
#### Some notes on automatic negotiation:
You can do protocol negotiation, either by being told that HTTP/2 protocol is available using the `alt-svc` HTTP header, or by participating in TLS / ALPN negotiation. In any case, you only get told that HTTP/2 is available relatively late in the connection setup. Since the flows for HTTP/1.1 and HTTP/2 need to be wired differently, it is hard to negotiate on-the-fly and then wire things correctly in a pool setup. That means, you need to run a pre-flight request to figure out whether a certain host would support HTTP/2 and keep that information around for the rest of the session / application runtime. I would see that as an additional future feature. We could keep the http2 selection setting in `host-connection-pool` as selecting from `http1` / `http2` and then later add `auto` or `preflight`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "jrudolph (via GitHub)" <gi...@apache.org>.
jrudolph commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2056372271
A few thoughts here:
* One main point of HTTP/2 is that you only need a single TCP connection open and multiplex multiple requests on top of it. The benefit is to amortize the cost of setting up a connection (TLS and CWND setup). One drawback is that having a single connection is a potential clump risk, i.e. it may stay open longer than intended (load balancing changes), any TCP connection state problems will affect all ongoing (and potentially future) requests.
* The main point of the HTTP/1.1 `singleRequest` / pool API is to relieve the user of the HTTP connection handling. With HTTP/2 this is less of a concern as a single connection can already support multiple ongoing HTTP requests at the same time.
What we already have for HTTP/2:
* create a single connection and multiplex requests over it using `OutgoingConnection.http2` (returns a `Flow`)
* create a "managed connection" which automatically reconnects an HTTP2 connection when the connection fails using `OutgoingConnection.managedPersistentHttp2` (returns a `Flow`)
What is missing:
* "pool" for HTTP/2 that keeps a single managed persistent HTTP/2 connection per host and automatically dispatches requests based on the host to that connection (that relieves code using the client from keeping state)
* `singleRequest`-like APIs using that pool (or on top of one of the existing `Flow` returning solutions)
* any kind of automatic negotiation of whether to use HTTP/1.1 or HTTP/2
In terms of essential functionality, it seems that not much is missing. Many backend usages that need an HTTP/2 client are already supported if you can do the "dispatch to the right host connection" (which only requires that you can keep the single managed connection as a state in your client shim).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "jrudolph (via GitHub)" <gi...@apache.org>.
jrudolph commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2056542504
> `OutgoingConnection.managedPersistentHttp2` and `NewHostConnectionPool` are compatible enough for it to work
Which is not quite the case right now. `NewHostConnectionPool` is `Flow[RequestContext, ResponseContext]` while `managedPersistentHttp2` is `Flow[HttpRequest, HttpResponse]`. That's probably not a blocker, it's more an artifact of the HTTP/1.1 low-level impl not (yet) supporting the more general `RequestResponseAssociation` protocol to associated requests with responses that HTTP/2 supports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "jrudolph (via GitHub)" <gi...@apache.org>.
jrudolph commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2056372703
> I think we can proceed as follow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] support making HTTP client requests using HTTP/2 [pekko-http]
Posted by "jrudolph (via GitHub)" <gi...@apache.org>.
jrudolph commented on issue #484:
URL: https://github.com/apache/pekko-http/issues/484#issuecomment-2056417185
> What needs to be done when HTTP/2 is selected is to replace the code at `PoolInterface.apply`, to create an alternative connection flow that uses a single managed persistent HTTP/2 connection instead of the pool of HTTP/1.1 connections.
@samueleresca this is the first thing I would try. Hopefully, the flows implemented by `OutgoingConnection.managedPersistentHttp2` and `NewHostConnectionPool` are compatible enough for it to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org