You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Ryan Schmitt <rs...@apache.org> on 2025/10/11 02:32:14 UTC
Client 5.6 test findings
I ran our internal integration test suite against the current core+client
commits. I didn't find much, just two notes:
1. The async client has started decompressing responses. I had to add a
call to `.disableContentCompression()`. This API is new in 5.6. This may be
an issue for libraries that use the async client, since it makes it harder
to "straddle" multiple versions of the client and obtain consistent
behavior from all of them.
2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
versions, etc) started failing. I had to fix this by adding a
`HostnameVerificationPolicy.CLIENT` argument to the
`DefaultClientTlsStrategy` constructor call, in addition to the custom
`HostnameVerifier` I am already supplying. I think this must have been
caused by a constructor delegation change in `DefaultClientTlsStrategy`,
where the default policy was changed from `CLIENT` to `null` (which later
gets turned into `BOTH`).
Re: Client 5.6 test findings
Posted by Arturo Bernal <ab...@apache.org>.
Hi Ryan,
OFFLOCK enablement for the connection pool is underway.
Arturo
On Sat, 11 Oct 2025 at 5:36 PM, Ryan Schmitt <rs...@apache.org> wrote:
> I can also test the OFFLOCK connection pool once the pooling conn managers
> add support for it.
>
> On Fri, Oct 10, 2025 at 7:32 PM Ryan Schmitt <rs...@apache.org> wrote:
>
> > I ran our internal integration test suite against the current core+client
> > commits. I didn't find much, just two notes:
> >
> > 1. The async client has started decompressing responses. I had to add a
> > call to `.disableContentCompression()`. This API is new in 5.6. This may
> be
> > an issue for libraries that use the async client, since it makes it
> harder
> > to "straddle" multiple versions of the client and obtain consistent
> > behavior from all of them.
> > 2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
> > versions, etc) started failing. I had to fix this by adding a
> > `HostnameVerificationPolicy.CLIENT` argument to the
> > `DefaultClientTlsStrategy` constructor call, in addition to the custom
> > `HostnameVerifier` I am already supplying. I think this must have been
> > caused by a constructor delegation change in `DefaultClientTlsStrategy`,
> > where the default policy was changed from `CLIENT` to `null` (which later
> > gets turned into `BOTH`).
> >
>
Re: Client 5.6 test findings
Posted by Ryan Schmitt <rs...@apache.org>.
I can also test the OFFLOCK connection pool once the pooling conn managers
add support for it.
On Fri, Oct 10, 2025 at 7:32 PM Ryan Schmitt <rs...@apache.org> wrote:
> I ran our internal integration test suite against the current core+client
> commits. I didn't find much, just two notes:
>
> 1. The async client has started decompressing responses. I had to add a
> call to `.disableContentCompression()`. This API is new in 5.6. This may be
> an issue for libraries that use the async client, since it makes it harder
> to "straddle" multiple versions of the client and obtain consistent
> behavior from all of them.
> 2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
> versions, etc) started failing. I had to fix this by adding a
> `HostnameVerificationPolicy.CLIENT` argument to the
> `DefaultClientTlsStrategy` constructor call, in addition to the custom
> `HostnameVerifier` I am already supplying. I think this must have been
> caused by a constructor delegation change in `DefaultClientTlsStrategy`,
> where the default policy was changed from `CLIENT` to `null` (which later
> gets turned into `BOTH`).
>
Re: Client 5.6 test findings
Posted by Arturo Bernal <ab...@apache.org>.
I’ll take a look on this later today
Arturo
On Sun, 12 Oct 2025 at 12:07 AM, Ryan Schmitt <rs...@apache.org> wrote:
> > Please raise a JIRA if you think there is a defect.
>
> I think the behavior is arbitrary. There's no way tell what the hostname
> verification policy should be, based solely on the presence of a
> non-default `HostnameVerifier`, because there's no way of knowing whether
> the supplied verifier is stricter than the built-in one (in which case BOTH
> is a good default) or more permissive (in which case CLIENT is the right
> behavior).
>
> Anyway, I found some actual problems. The first is that we have two Brotli
> dependencies, one of which is marked as optional and one which is not:
>
> <dependency>
> <groupId>org.brotli</groupId>
> <artifactId>dec</artifactId>
> <optional>true</optional>
> </dependency>
> <dependency>
> <groupId>com.aayushatharva.brotli4j</groupId>
> <artifactId>brotli4j</artifactId>
> </dependency>
>
> Can we take this opportunity to get rid of BrotliInputStreamFactory and
> drop the dependency on org.brotli:dec? It hasn't had a release since 2017.
>
> The other issue is that our commons-compress dependency is marked as
> optional, but we import its classes as if it weren't:
>
>
>
> httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CommonsCompressingEntity.java
> 35:import org.apache.commons.compress.compressors.CompressorException;
> 36:import
> org.apache.commons.compress.compressors.CompressorStreamFactory;
>
>
>
> httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CommonsCompressDecoderFactory.java
> 37:import org.apache.commons.compress.compressors.CompressorException;
> 38:import
> org.apache.commons.compress.compressors.CompressorStreamFactory;
>
> This results in ContentCodecRegistry throwing
> an ExceptionInInitializerError at runtime. The issue here seems to be the
> `catch` clause on `CompressorException`. We should probably just dial that
> up to `IOException`.
>
> On Sat, Oct 11, 2025 at 11:47 AM Oleg Kalnichevski <ol...@apache.org>
> wrote:
>
> >
> >
> > On 10/11/2025 4:32 AM, Ryan Schmitt wrote:
> > > I ran our internal integration test suite against the current
> core+client
> > > commits. I didn't find much, just two notes:
> > >
> > > 1. The async client has started decompressing responses. I had to add a
> > > call to `.disableContentCompression()`. This API is new in 5.6. This
> may
> > be
> > > an issue for libraries that use the async client, since it makes it
> > harder
> > > to "straddle" multiple versions of the client and obtain consistent
> > > behavior from all of them.
> >
> > This is a new feature in a feature release. Feature releases can have
> > behavioral differences with earlier versions. There is going to be a
> > note in the release notes regarding this change.
> >
> > > 2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
> > > versions, etc) started failing. I had to fix this by adding a
> > > `HostnameVerificationPolicy.CLIENT` argument to the
> > > `DefaultClientTlsStrategy` constructor call, in addition to the custom
> > > `HostnameVerifier` I am already supplying. I think this must have been
> > > caused by a constructor delegation change in
> `DefaultClientTlsStrategy`,
> > > where the default policy was changed from `CLIENT` to `null` (which
> later
> > > gets turned into `BOTH`).
> > >
> >
> > Please raise a JIRA if you think there is a defect.
> >
> > Oleg
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
> >
>
Re: Client 5.6 test findings
Posted by Ryan Schmitt <rs...@apache.org>.
I haven't spotted a problem with the `#runtimeAvailable` method; it's
actually the `#decoder(String)` method in the same class that is triggering
the loading of commons-compress classes.
The best policy is to not import any classes at all from an optional
dependency, and to instead use only reflection. The JLS and JVMS make no
guarantees about lazy classloading.
On Sun, Oct 12, 2025 at 12:47 AM Oleg Kalnichevski <ol...@apache.org> wrote:
>
>
> On 10/12/2025 12:06 AM, Ryan Schmitt wrote:
>
>
> > The other issue is that our commons-compress dependency is marked as
> > optional, but we import its classes as if it weren't:
> >
> >
> That should not be a problem. There is a safe-guard check that ensures
> the factory is loaded from the classpath only if the optional dependency
> is also present.
>
> ```
> if
> (CommonsCompressDecoderFactory.runtimeAvailable(c.token())) {
> m.put(c, new Codec(
> e -> new CommonsCompressingEntity(e,
> c.token()),
> ent -> new DecompressingEntity(ent,
>
> CommonsCompressDecoderFactory.decoder(c.token()))));
> }
> ```
>
> The CommonsCompressDecoderFactory#runtimeAvailable implementation looks
> wrong though. I overlooked it in my review.
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: Client 5.6 test findings
Posted by Oleg Kalnichevski <ol...@apache.org>.
On 10/12/2025 12:06 AM, Ryan Schmitt wrote:
> The other issue is that our commons-compress dependency is marked as
> optional, but we import its classes as if it weren't:
>
>
That should not be a problem. There is a safe-guard check that ensures
the factory is loaded from the classpath only if the optional dependency
is also present.
```
if
(CommonsCompressDecoderFactory.runtimeAvailable(c.token())) {
m.put(c, new Codec(
e -> new CommonsCompressingEntity(e,
c.token()),
ent -> new DecompressingEntity(ent,
CommonsCompressDecoderFactory.decoder(c.token()))));
}
```
The CommonsCompressDecoderFactory#runtimeAvailable implementation looks
wrong though. I overlooked it in my review.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Client 5.6 test findings
Posted by Ryan Schmitt <rs...@apache.org>.
> Please raise a JIRA if you think there is a defect.
I think the behavior is arbitrary. There's no way tell what the hostname
verification policy should be, based solely on the presence of a
non-default `HostnameVerifier`, because there's no way of knowing whether
the supplied verifier is stricter than the built-in one (in which case BOTH
is a good default) or more permissive (in which case CLIENT is the right
behavior).
Anyway, I found some actual problems. The first is that we have two Brotli
dependencies, one of which is marked as optional and one which is not:
<dependency>
<groupId>org.brotli</groupId>
<artifactId>dec</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.aayushatharva.brotli4j</groupId>
<artifactId>brotli4j</artifactId>
</dependency>
Can we take this opportunity to get rid of BrotliInputStreamFactory and
drop the dependency on org.brotli:dec? It hasn't had a release since 2017.
The other issue is that our commons-compress dependency is marked as
optional, but we import its classes as if it weren't:
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CommonsCompressingEntity.java
35:import org.apache.commons.compress.compressors.CompressorException;
36:import
org.apache.commons.compress.compressors.CompressorStreamFactory;
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CommonsCompressDecoderFactory.java
37:import org.apache.commons.compress.compressors.CompressorException;
38:import
org.apache.commons.compress.compressors.CompressorStreamFactory;
This results in ContentCodecRegistry throwing
an ExceptionInInitializerError at runtime. The issue here seems to be the
`catch` clause on `CompressorException`. We should probably just dial that
up to `IOException`.
On Sat, Oct 11, 2025 at 11:47 AM Oleg Kalnichevski <ol...@apache.org> wrote:
>
>
> On 10/11/2025 4:32 AM, Ryan Schmitt wrote:
> > I ran our internal integration test suite against the current core+client
> > commits. I didn't find much, just two notes:
> >
> > 1. The async client has started decompressing responses. I had to add a
> > call to `.disableContentCompression()`. This API is new in 5.6. This may
> be
> > an issue for libraries that use the async client, since it makes it
> harder
> > to "straddle" multiple versions of the client and obtain consistent
> > behavior from all of them.
>
> This is a new feature in a feature release. Feature releases can have
> behavioral differences with earlier versions. There is going to be a
> note in the release notes regarding this change.
>
> > 2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
> > versions, etc) started failing. I had to fix this by adding a
> > `HostnameVerificationPolicy.CLIENT` argument to the
> > `DefaultClientTlsStrategy` constructor call, in addition to the custom
> > `HostnameVerifier` I am already supplying. I think this must have been
> > caused by a constructor delegation change in `DefaultClientTlsStrategy`,
> > where the default policy was changed from `CLIENT` to `null` (which later
> > gets turned into `BOTH`).
> >
>
> Please raise a JIRA if you think there is a defect.
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: Client 5.6 test findings
Posted by Oleg Kalnichevski <ol...@apache.org>.
On 10/11/2025 4:32 AM, Ryan Schmitt wrote:
> I ran our internal integration test suite against the current core+client
> commits. I didn't find much, just two notes:
>
> 1. The async client has started decompressing responses. I had to add a
> call to `.disableContentCompression()`. This API is new in 5.6. This may be
> an issue for libraries that use the async client, since it makes it harder
> to "straddle" multiple versions of the client and obtain consistent
> behavior from all of them.
This is a new feature in a feature release. Feature releases can have
behavioral differences with earlier versions. There is going to be a
note in the release notes regarding this change.
> 2. Our tests for insecure mode SSL (self-signed certs, obsolete TLS
> versions, etc) started failing. I had to fix this by adding a
> `HostnameVerificationPolicy.CLIENT` argument to the
> `DefaultClientTlsStrategy` constructor call, in addition to the custom
> `HostnameVerifier` I am already supplying. I think this must have been
> caused by a constructor delegation change in `DefaultClientTlsStrategy`,
> where the default policy was changed from `CLIENT` to `null` (which later
> gets turned into `BOTH`).
>
Please raise a JIRA if you think there is a defect.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org