You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Ryan Schmitt <rs...@apache.org> on 2025/04/21 17:40:01 UTC
Unix domain socket support
I've got Apache client 5.4.3 working with a custom connection operator
that implements UDS support. I want to upstream this as a proper
feature, since it's significantly more efficient than localhost. I
want to discuss the design of this feature a bit; since it's a non-TCP
transport layer, we have to be careful about how it's integrated into
the overall design of the client.
CONVENTION
There are a couple of ways clients expose UDS support. One is to use a
URI scheme like unix or http+unix and provide the path to the socket
as the URI authority:
http+unix://%2Fpath%2Fto%2Fsocket/socket.sock/path?query
This is a slight abuse of notation since Unix sockets are a transport,
rather than an application-level concern; additionally, neither scheme
has no IANA registration.
Another solution is to provide the UDS path out-of-band, as in this
example from curl's man page:
curl --unix-socket socket-path https://example.com
This is cleaner since it gives us a principled way to derive the Host
header (typically localhost) and doesn't stuff transport details into
the URI. It's also easier for end users to convert an existing
localhost call to UDS. I think this is my preferred approach. The main
question is which of the many config types should expose this option.
RequestConfig seems reasonable, but there's also ConnectionConfig and
SocketConfig.
IMPLEMENTATION
There are two options here: the JDK16 standard library, and
JUnixSocket. I think we should do basically what we did for ALPN
before JDK 1.8 Maintenance Release 3: model JUnixSocket as an optional
dependency, use reflection to check for both implementations, and fail
if neither one can be found. For JDK16 support we have the option of
vending a multi-release jar, but this would add significant build-time
complexity with little clear benefit over a MethodHandle set in a
static initializer (this approach has equivalent performance to static
linkage, for example).
ASYNC
According to my research, JDK16 and JUnixSocket both support
non-blocking IO through the usual NIO abstractions like SocketChannel
and Selector.
SOCKET TYPES
There are two types of Unix domain sockets. AF_UNIX is supported
pretty much everywhere (even Windows) and is accessed as a file on the
filesystem. Linux additionally offers abstract Unix domain sockets,
which are accessed through a special namespace. Only AF_UNIX is
supported by JDK16, so this feature would require JUnixSocket. I
recommend only supporting AF_UNIX for the time being, since I'm not
aware of any other HttpComponents features that are non-portable (such
as epoll support).
INTERACTIONS
UDS support would need to work correctly with connection pooling,
route modeling, and existing features like proxy support.
Additionally, any code that assumes that the remote address is an
InetAddress or InetSocketAddress may need to be adjusted. Finally,
features like request logging and debug logging may need to account
for the possibility of a UDS connection.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Arturo Bernal <ab...@apache.org>.
Hi Ryan,
A concrete driver for AF_VSOCK is the microVM / confidential-computing
space (e.g., Nitro Enclaves) where vsock is the standard host↔guest
channel, and also Kata Containers where vsock is used for host/agent comms.
Thanks,
Arturo
On Sat, Feb 14, 2026 at 12:48 PM Michael Osipov <mi...@apache.org> wrote:
> Guys, not forget AF_HYPERV :-D
>
> Sample impl from Microsoft:
> https://github.com/freebsd/freebsd-src/blob/625dc44832cd760be3d7242d8e21a530c7e32bfc/sys/dev/hyperv/hvsock/hv_sock.c#L110
>
> On 2026/02/13 16:36:42 Ryan Schmitt wrote:
> > There are lots of exotic socket types, including various non-portable
> > subtypes of Unix domain sockets. What are people actually asking for? In
> > the docker-java client for example, we found... aftermarket support for
> > both Unix domain sockets and named pipes (as a kind of ersatz UDS for
> older
> > versions of Windows), but no AF_VSOCK.
> >
> > On Thu, Feb 12, 2026 at 10:38 AM Arturo Bernal <ab...@apache.org>
> wrote:
> >
> > > Hi Ryan,
> > >
> > > Now that UDS support is in, do you think it makes sense to also
> upstream
> > > AF_VSOCK as an optional follow-up?
> > >
> > > I’m thinking: same route/pool-key + custom connection-operator
> pattern, but
> > > gated behind a builder flag and backed by JUnixSocket’s vsock module
> (so
> > > “JUnixSocket-only”, Linux/VM focused).
> > >
> > > Thanks,
> > >
> > >
> > > On Fri, May 30, 2025 at 9:33 PM Ryan Schmitt <rs...@pobox.com>
> wrote:
> > >
> > > > OK, I'll do it.
> > > >
> > > > On Fri, May 30, 2025 at 2:27 AM Oleg Kalnichevski <ol...@apache.org>
> > > > wrote:
> > > > >
> > > > > On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> > > > > > Do we have any existing integration test coverage for socket and
> > > > > > connection timeouts? I only see mock-based unit tests.
> > > > > >
> > > > >
> > > > > We do not. Timeouts are quite difficult to test reliably and up to
> now
> > > > > no one stepped forth to put efforts into building them.
> > > > >
> > > > > Oleg
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > > > > For additional commands, e-mail: dev-help@hc.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > > > For additional commands, e-mail: dev-help@hc.apache.org
> > > >
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: Unix domain socket support
Posted by Michael Osipov <mi...@apache.org>.
Guys, not forget AF_HYPERV :-D
Sample impl from Microsoft: https://github.com/freebsd/freebsd-src/blob/625dc44832cd760be3d7242d8e21a530c7e32bfc/sys/dev/hyperv/hvsock/hv_sock.c#L110
On 2026/02/13 16:36:42 Ryan Schmitt wrote:
> There are lots of exotic socket types, including various non-portable
> subtypes of Unix domain sockets. What are people actually asking for? In
> the docker-java client for example, we found... aftermarket support for
> both Unix domain sockets and named pipes (as a kind of ersatz UDS for older
> versions of Windows), but no AF_VSOCK.
>
> On Thu, Feb 12, 2026 at 10:38 AM Arturo Bernal <ab...@apache.org> wrote:
>
> > Hi Ryan,
> >
> > Now that UDS support is in, do you think it makes sense to also upstream
> > AF_VSOCK as an optional follow-up?
> >
> > I’m thinking: same route/pool-key + custom connection-operator pattern, but
> > gated behind a builder flag and backed by JUnixSocket’s vsock module (so
> > “JUnixSocket-only”, Linux/VM focused).
> >
> > Thanks,
> >
> >
> > On Fri, May 30, 2025 at 9:33 PM Ryan Schmitt <rs...@pobox.com> wrote:
> >
> > > OK, I'll do it.
> > >
> > > On Fri, May 30, 2025 at 2:27 AM Oleg Kalnichevski <ol...@apache.org>
> > > wrote:
> > > >
> > > > On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> > > > > Do we have any existing integration test coverage for socket and
> > > > > connection timeouts? I only see mock-based unit tests.
> > > > >
> > > >
> > > > We do not. Timeouts are quite difficult to test reliably and up to now
> > > > no one stepped forth to put efforts into building them.
> > > >
> > > > Oleg
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > > > For additional commands, e-mail: dev-help@hc.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: dev-help@hc.apache.org
> > >
> > >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Ryan Schmitt <rs...@apache.org>.
There are lots of exotic socket types, including various non-portable
subtypes of Unix domain sockets. What are people actually asking for? In
the docker-java client for example, we found... aftermarket support for
both Unix domain sockets and named pipes (as a kind of ersatz UDS for older
versions of Windows), but no AF_VSOCK.
On Thu, Feb 12, 2026 at 10:38 AM Arturo Bernal <ab...@apache.org> wrote:
> Hi Ryan,
>
> Now that UDS support is in, do you think it makes sense to also upstream
> AF_VSOCK as an optional follow-up?
>
> I’m thinking: same route/pool-key + custom connection-operator pattern, but
> gated behind a builder flag and backed by JUnixSocket’s vsock module (so
> “JUnixSocket-only”, Linux/VM focused).
>
> Thanks,
>
>
> On Fri, May 30, 2025 at 9:33 PM Ryan Schmitt <rs...@pobox.com> wrote:
>
> > OK, I'll do it.
> >
> > On Fri, May 30, 2025 at 2:27 AM Oleg Kalnichevski <ol...@apache.org>
> > wrote:
> > >
> > > On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> > > > Do we have any existing integration test coverage for socket and
> > > > connection timeouts? I only see mock-based unit tests.
> > > >
> > >
> > > We do not. Timeouts are quite difficult to test reliably and up to now
> > > no one stepped forth to put efforts into building them.
> > >
> > > Oleg
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: dev-help@hc.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
> >
>
Re: Unix domain socket support
Posted by Arturo Bernal <ab...@apache.org>.
Hi Ryan,
Now that UDS support is in, do you think it makes sense to also upstream
AF_VSOCK as an optional follow-up?
I’m thinking: same route/pool-key + custom connection-operator pattern, but
gated behind a builder flag and backed by JUnixSocket’s vsock module (so
“JUnixSocket-only”, Linux/VM focused).
Thanks,
On Fri, May 30, 2025 at 9:33 PM Ryan Schmitt <rs...@pobox.com> wrote:
> OK, I'll do it.
>
> On Fri, May 30, 2025 at 2:27 AM Oleg Kalnichevski <ol...@apache.org>
> wrote:
> >
> > On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> > > Do we have any existing integration test coverage for socket and
> > > connection timeouts? I only see mock-based unit tests.
> > >
> >
> > We do not. Timeouts are quite difficult to test reliably and up to now
> > no one stepped forth to put efforts into building them.
> >
> > Oleg
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: Unix domain socket support
Posted by Ryan Schmitt <rs...@pobox.com>.
OK, I'll do it.
On Fri, May 30, 2025 at 2:27 AM Oleg Kalnichevski <ol...@apache.org> wrote:
>
> On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> > Do we have any existing integration test coverage for socket and
> > connection timeouts? I only see mock-based unit tests.
> >
>
> We do not. Timeouts are quite difficult to test reliably and up to now
> no one stepped forth to put efforts into building them.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2025-05-29 at 15:44 -0700, Ryan Schmitt wrote:
> Do we have any existing integration test coverage for socket and
> connection timeouts? I only see mock-based unit tests.
>
We do not. Timeouts are quite difficult to test reliably and up to now
no one stepped forth to put efforts into building them.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Ryan Schmitt <rs...@apache.org>.
Do we have any existing integration test coverage for socket and
connection timeouts? I only see mock-based unit tests.
On Fri, Apr 25, 2025 at 1:04 PM Oleg Kalnichevski <ol...@apache.org> wrote:
>
> On Thu, 2025-04-24 at 13:04 -0700, Ryan Schmitt wrote:
> > > UDS looks like a route / connection property, not a request one. So
> > > RequestConfig would likely be a bad choice. ConnectionConfig would
> > > make
> > > a better choice in my option. Better yet, it could be a method, say
> > > #useUnixSocket, in the `PoolingHttpClientConnectionManagerBuilder`
> > > that
> > > would make the connection manager use a special UDS based
> > > `HttpClientConnectionOperator`. This would be pretty much
> > > equivalent to
> > > the `curl --unix-socket` option.
> >
> > I think that a Unix domain socket is conceptually a proxy: you're
> > connecting to `localhost` but proxying the request through an IPC
> > mechanism. And I think it's awkward to say that a route property is
> > not a request property for two reasons:
> >
> > 1. There is currently a `proxy` field defined on `RequestConfig`
> > (admittedly deprecated)
> > 2. The `HttpRoutePlanner` computes the route entirely from
> > request-scoped information: the target, the HttpContext, and the
> > HttpRequest itself.
>
> Hi Ryan
>
> You are right. The route for a request gets determined based on
> properties of the request or some applicable defaults.
>
> I am fine with using `RequestConfig` to contain UDS properties as well.
> In this case we should also un-deprecate the `proxy` property in
> `RequestConfig` to be consistent.
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2025-04-24 at 13:04 -0700, Ryan Schmitt wrote:
> > UDS looks like a route / connection property, not a request one. So
> > RequestConfig would likely be a bad choice. ConnectionConfig would
> > make
> > a better choice in my option. Better yet, it could be a method, say
> > #useUnixSocket, in the `PoolingHttpClientConnectionManagerBuilder`
> > that
> > would make the connection manager use a special UDS based
> > `HttpClientConnectionOperator`. This would be pretty much
> > equivalent to
> > the `curl --unix-socket` option.
>
> I think that a Unix domain socket is conceptually a proxy: you're
> connecting to `localhost` but proxying the request through an IPC
> mechanism. And I think it's awkward to say that a route property is
> not a request property for two reasons:
>
> 1. There is currently a `proxy` field defined on `RequestConfig`
> (admittedly deprecated)
> 2. The `HttpRoutePlanner` computes the route entirely from
> request-scoped information: the target, the HttpContext, and the
> HttpRequest itself.
Hi Ryan
You are right. The route for a request gets determined based on
properties of the request or some applicable defaults.
I am fine with using `RequestConfig` to contain UDS properties as well.
In this case we should also un-deprecate the `proxy` property in
`RequestConfig` to be consistent.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Ryan Schmitt <rs...@apache.org>.
> UDS looks like a route / connection property, not a request one. So
> RequestConfig would likely be a bad choice. ConnectionConfig would make
> a better choice in my option. Better yet, it could be a method, say
> #useUnixSocket, in the `PoolingHttpClientConnectionManagerBuilder` that
> would make the connection manager use a special UDS based
> `HttpClientConnectionOperator`. This would be pretty much equivalent to
> the `curl --unix-socket` option.
I think that a Unix domain socket is conceptually a proxy: you're
connecting to `localhost` but proxying the request through an IPC
mechanism. And I think it's awkward to say that a route property is
not a request property for two reasons:
1. There is currently a `proxy` field defined on `RequestConfig`
(admittedly deprecated)
2. The `HttpRoutePlanner` computes the route entirely from
request-scoped information: the target, the HttpContext, and the
HttpRequest itself.
Another issue with modeling a UDS as a proxy is that `HttpHost` and
its superinterface `NamedEndpoint` are both coupled to TCP/IP and DNS
(they're built around concepts like hostname, InetAddress, and port
number). I see why you want to model UDS as an attribute directly on
`HttpRoute`, since `ConnectionConfig` is a function of the
`HttpRoute`. We could add a field like `private final Path
unixDomainSocket`, but I still think there's an ergonomic issue here
with requiring a custom `HttpRoutePlanner` instead of offering
declarative configuration of UDS.
The way forward here might be to expose this as an option on
`DefaultRoutePlanner`. Users would create a UDS-enabled instance which
they would then supply directly to the client builder. This would then
produce a UDS-enabled `HttpRoute` for UDS-enabled requests (TODO:
enabled based on what?), and the connection operator would then
connect directly to the Unix socket by calling the relevant low-level
code in httpcore5.
Personally, I still think that `RequestConfig` is a reasonable way of
exposing this option.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2025-04-21 at 10:40 -0700, Ryan Schmitt wrote:
> I've got Apache client 5.4.3 working with a custom connection
> operator
> that implements UDS support. I want to upstream this as a proper
> feature, since it's significantly more efficient than localhost. I
> want to discuss the design of this feature a bit; since it's a non-
> TCP
> transport layer, we have to be careful about how it's integrated into
> the overall design of the client.
>
> CONVENTION
>
> There are a couple of ways clients expose UDS support. One is to use
> a
> URI scheme like unix or http+unix and provide the path to the socket
> as the URI authority:
>
> http+unix://%2Fpath%2Fto%2Fsocket/socket.sock/path?query
>
> This is a slight abuse of notation since Unix sockets are a
> transport,
> rather than an application-level concern; additionally, neither
> scheme
> has no IANA registration.
>
> Another solution is to provide the UDS path out-of-band, as in this
> example from curl's man page:
>
> curl --unix-socket socket-path https://example.com
>
> This is cleaner since it gives us a principled way to derive the Host
> header (typically localhost) and doesn't stuff transport details into
> the URI. It's also easier for end users to convert an existing
> localhost call to UDS. I think this is my preferred approach. The
> main
> question is which of the many config types should expose this option.
> RequestConfig seems reasonable, but there's also ConnectionConfig and
> SocketConfig.
>
Hi Ryan
My apologies for not having responded sooner.
UDS looks like a route / connection property, not a request one. So
RequestConfig would likely be a bad choice. ConnectionConfig would make
a better choice in my option. Better yet, it could be a method, say
#useUnixSocket, in the `PoolingHttpClientConnectionManagerBuilder` that
would make the connection manager use a special UDS based
`HttpClientConnectionOperator`. This would be pretty much equivalent to
the `curl --unix-socket` option.
> ASYNC
>
> According to my research, JDK16 and JUnixSocket both support
> non-blocking IO through the usual NIO abstractions like SocketChannel
> and Selector.
>
This will likely require changes to the i/o reactor code in core but
should be doable.
> INTERACTIONS
>
> UDS support would need to work correctly with connection pooling,
> route modeling, and existing features like proxy support.
> Additionally, any code that assumes that the remote address is an
> InetAddress or InetSocketAddress may need to be adjusted. Finally,
> features like request logging and debug logging may need to account
> for the possibility of a UDS connection.
>
As far as I know InetAddress / InetSocketAddress should be exclusively
used by connection operators, so we are safe. HttpRoute may need an
extra attribute though.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Michael Osipov <mi...@apache.org>.
On 2025/04/21 17:40:01 Ryan Schmitt wrote:
> I've got Apache client 5.4.3 working with a custom connection operator
> that implements UDS support. I want to upstream this as a proper
> feature, since it's significantly more efficient than localhost. I
> want to discuss the design of this feature a bit; since it's a non-TCP
> transport layer, we have to be careful about how it's integrated into
> the overall design of the client.
>
> CONVENTION
>
> There are a couple of ways clients expose UDS support. One is to use a
> URI scheme like unix or http+unix and provide the path to the socket
> as the URI authority:
>
> http+unix://%2Fpath%2Fto%2Fsocket/socket.sock/path?query
>
> This is a slight abuse of notation since Unix sockets are a transport,
> rather than an application-level concern; additionally, neither scheme
> has no IANA registration.
>
> Another solution is to provide the UDS path out-of-band, as in this
> example from curl's man page:
>
> curl --unix-socket socket-path https://example.com
>
> This is cleaner since it gives us a principled way to derive the Host
> header (typically localhost) and doesn't stuff transport details into
> the URI. It's also easier for end users to convert an existing
> localhost call to UDS. I think this is my preferred approach. The main
> question is which of the many config types should expose this option.
> RequestConfig seems reasonable, but there's also ConnectionConfig and
> SocketConfig.
>
> IMPLEMENTATION
>
> There are two options here: the JDK16 standard library, and
> JUnixSocket. I think we should do basically what we did for ALPN
> before JDK 1.8 Maintenance Release 3: model JUnixSocket as an optional
> dependency, use reflection to check for both implementations, and fail
> if neither one can be found. For JDK16 support we have the option of
> vending a multi-release jar, but this would add significant build-time
> complexity with little clear benefit over a MethodHandle set in a
> static initializer (this approach has equivalent performance to static
> linkage, for example).
>
> ASYNC
>
> According to my research, JDK16 and JUnixSocket both support
> non-blocking IO through the usual NIO abstractions like SocketChannel
> and Selector.
>
> SOCKET TYPES
>
> There are two types of Unix domain sockets. AF_UNIX is supported
> pretty much everywhere (even Windows) and is accessed as a file on the
> filesystem. Linux additionally offers abstract Unix domain sockets,
> which are accessed through a special namespace. Only AF_UNIX is
> supported by JDK16, so this feature would require JUnixSocket. I
> recommend only supporting AF_UNIX for the time being, since I'm not
> aware of any other HttpComponents features that are non-portable (such
> as epoll support).
>
> INTERACTIONS
>
> UDS support would need to work correctly with connection pooling,
> route modeling, and existing features like proxy support.
> Additionally, any code that assumes that the remote address is an
> InetAddress or InetSocketAddress may need to be adjusted. Finally,
> features like request logging and debug logging may need to account
> for the possibility of a UDS connection.
Well-known issue: https://github.com/whatwg/url/issues/577
see also: https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass
I prefer JUnixSocket since it works everywhere and has been around for years AND I prefer the path to the socket be out of band to avoid confusing and interpretation.
Do you have competitive numbers how the speed up looks for you?
M
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Ryan Schmitt <rs...@apache.org>.
UDS = Unix Domain Socket
On Tue, Apr 22, 2025 at 2:07 PM Gary Gregory <ga...@gmail.com> wrote:
>
> Hi all,
>
> I'm not sure how to ask this question, so I'll be straightforward: If
> UDS is https://en.wikipedia.org/wiki/Unified_Diagnostic_Services, why
> should it be in an HTTP-focused project?
>
> I might have missed some context in a previous message of course.
>
> Gary
>
> On Mon, Apr 21, 2025 at 1:40 PM Ryan Schmitt <rs...@apache.org> wrote:
> >
> > I've got Apache client 5.4.3 working with a custom connection operator
> > that implements UDS support. I want to upstream this as a proper
> > feature, since it's significantly more efficient than localhost. I
> > want to discuss the design of this feature a bit; since it's a non-TCP
> > transport layer, we have to be careful about how it's integrated into
> > the overall design of the client.
> >
> > CONVENTION
> >
> > There are a couple of ways clients expose UDS support. One is to use a
> > URI scheme like unix or http+unix and provide the path to the socket
> > as the URI authority:
> >
> > http+unix://%2Fpath%2Fto%2Fsocket/socket.sock/path?query
> >
> > This is a slight abuse of notation since Unix sockets are a transport,
> > rather than an application-level concern; additionally, neither scheme
> > has no IANA registration.
> >
> > Another solution is to provide the UDS path out-of-band, as in this
> > example from curl's man page:
> >
> > curl --unix-socket socket-path https://example.com
> >
> > This is cleaner since it gives us a principled way to derive the Host
> > header (typically localhost) and doesn't stuff transport details into
> > the URI. It's also easier for end users to convert an existing
> > localhost call to UDS. I think this is my preferred approach. The main
> > question is which of the many config types should expose this option.
> > RequestConfig seems reasonable, but there's also ConnectionConfig and
> > SocketConfig.
> >
> > IMPLEMENTATION
> >
> > There are two options here: the JDK16 standard library, and
> > JUnixSocket. I think we should do basically what we did for ALPN
> > before JDK 1.8 Maintenance Release 3: model JUnixSocket as an optional
> > dependency, use reflection to check for both implementations, and fail
> > if neither one can be found. For JDK16 support we have the option of
> > vending a multi-release jar, but this would add significant build-time
> > complexity with little clear benefit over a MethodHandle set in a
> > static initializer (this approach has equivalent performance to static
> > linkage, for example).
> >
> > ASYNC
> >
> > According to my research, JDK16 and JUnixSocket both support
> > non-blocking IO through the usual NIO abstractions like SocketChannel
> > and Selector.
> >
> > SOCKET TYPES
> >
> > There are two types of Unix domain sockets. AF_UNIX is supported
> > pretty much everywhere (even Windows) and is accessed as a file on the
> > filesystem. Linux additionally offers abstract Unix domain sockets,
> > which are accessed through a special namespace. Only AF_UNIX is
> > supported by JDK16, so this feature would require JUnixSocket. I
> > recommend only supporting AF_UNIX for the time being, since I'm not
> > aware of any other HttpComponents features that are non-portable (such
> > as epoll support).
> >
> > INTERACTIONS
> >
> > UDS support would need to work correctly with connection pooling,
> > route modeling, and existing features like proxy support.
> > Additionally, any code that assumes that the remote address is an
> > InetAddress or InetSocketAddress may need to be adjusted. Finally,
> > features like request logging and debug logging may need to account
> > for the possibility of a UDS connection.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Unix domain socket support
Posted by Gary Gregory <ga...@gmail.com>.
Hi all,
I'm not sure how to ask this question, so I'll be straightforward: If
UDS is https://en.wikipedia.org/wiki/Unified_Diagnostic_Services, why
should it be in an HTTP-focused project?
I might have missed some context in a previous message of course.
Gary
On Mon, Apr 21, 2025 at 1:40 PM Ryan Schmitt <rs...@apache.org> wrote:
>
> I've got Apache client 5.4.3 working with a custom connection operator
> that implements UDS support. I want to upstream this as a proper
> feature, since it's significantly more efficient than localhost. I
> want to discuss the design of this feature a bit; since it's a non-TCP
> transport layer, we have to be careful about how it's integrated into
> the overall design of the client.
>
> CONVENTION
>
> There are a couple of ways clients expose UDS support. One is to use a
> URI scheme like unix or http+unix and provide the path to the socket
> as the URI authority:
>
> http+unix://%2Fpath%2Fto%2Fsocket/socket.sock/path?query
>
> This is a slight abuse of notation since Unix sockets are a transport,
> rather than an application-level concern; additionally, neither scheme
> has no IANA registration.
>
> Another solution is to provide the UDS path out-of-band, as in this
> example from curl's man page:
>
> curl --unix-socket socket-path https://example.com
>
> This is cleaner since it gives us a principled way to derive the Host
> header (typically localhost) and doesn't stuff transport details into
> the URI. It's also easier for end users to convert an existing
> localhost call to UDS. I think this is my preferred approach. The main
> question is which of the many config types should expose this option.
> RequestConfig seems reasonable, but there's also ConnectionConfig and
> SocketConfig.
>
> IMPLEMENTATION
>
> There are two options here: the JDK16 standard library, and
> JUnixSocket. I think we should do basically what we did for ALPN
> before JDK 1.8 Maintenance Release 3: model JUnixSocket as an optional
> dependency, use reflection to check for both implementations, and fail
> if neither one can be found. For JDK16 support we have the option of
> vending a multi-release jar, but this would add significant build-time
> complexity with little clear benefit over a MethodHandle set in a
> static initializer (this approach has equivalent performance to static
> linkage, for example).
>
> ASYNC
>
> According to my research, JDK16 and JUnixSocket both support
> non-blocking IO through the usual NIO abstractions like SocketChannel
> and Selector.
>
> SOCKET TYPES
>
> There are two types of Unix domain sockets. AF_UNIX is supported
> pretty much everywhere (even Windows) and is accessed as a file on the
> filesystem. Linux additionally offers abstract Unix domain sockets,
> which are accessed through a special namespace. Only AF_UNIX is
> supported by JDK16, so this feature would require JUnixSocket. I
> recommend only supporting AF_UNIX for the time being, since I'm not
> aware of any other HttpComponents features that are non-portable (such
> as epoll support).
>
> INTERACTIONS
>
> UDS support would need to work correctly with connection pooling,
> route modeling, and existing features like proxy support.
> Additionally, any code that assumes that the remote address is an
> InetAddress or InetSocketAddress may need to be adjusted. Finally,
> features like request logging and debug logging may need to account
> for the possibility of a UDS connection.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org