Core vs client

[Ryan Schmitt <rs...@apache.org>]

I had a few basic questions about the division between core and client:

1. Why the split between core and client?

2. Are there consumers of core who don't also consume client?

3. What determines whether an abstraction lives in core or client? (For
example,

4. Does core contain any interfaces that are intended for client use only?

5. Why is there a dedicated set of client artifacts, but not server
artifacts?

6. Will :httpcore5-h2 be merged into :httpcore5 before release? If not, is
there a reason why it is being kept separate?

Re: Core vs client

[Oleg Kalnichevski <ol...@apache.org>]

On Fri, 2018-12-07 at 15:48 -0800, Ryan Schmitt wrote:
> Point of clarification: what exactly does @Internal mean? Does it
> mean that
> customers should not invoke methods on that type, or does it mean
> that they
> should not import it at all? (In the latter case, we have more
> control,
> since we can remove or merge types.)
> 

At the moment it is merely 'there be dragons' warning. It is up to us
to decide what that effectively implies. One option is to allow API
breakage of internal methods between minor releases (5.x.0).

Oleg


> On Wed, Dec 5, 2018 at 4:58 AM Oleg Kalnichevski <ol...@apache.org>
> wrote:
> 
> > On Tue, 2018-12-04 at 14:36 -0700, Gary Gregory wrote:
> > > On Tue, Dec 4, 2018 at 12:54 PM Ryan Schmitt <rschmitt@apache.org
> > > >
> > > wrote:
> > > 
> > > > My concern with the core/client split is that I think it
> > > > exposes
> > > > tons of
> > > > additional API surface area which will become a liability post-
> > > > release. My
> > > > recent changes for TLS handshake timeouts are a good example of
> > > > this: in
> > > > order to use the `connectTimeout` as the TLS handshake timeout
> > > > (which is
> > > > basically a bugfix), I had to make breaking API changes to over
> > > > a
> > > > dozen
> > > > public classes, and two interfaces as well (TlsStrategy
> > > > and TransportSecurityLayer).
> > > > 
> > > > The interface changes are particularly worrisome, because after
> > > > release
> > > > they'll be more or less impossible; we won't even have Java 8
> > > > default
> > > > methods to fall back on. Core defines over 130 public
> > > > interfaces,
> > > > about
> > > > half of which are new in 5.0, and only four of which are marked
> > > > @Internal,
> > > > so our compatibility obligations are clearly daunting.
> > > > 
> > > > Merging core into client is obviously an extreme suggestion,
> > > > but
> > > > I'm not
> > > > sure how else to attack the root of this problem.
> > > > 
> > > 
> > > No silver bullet here I am afraid.
> > > 
> > 
> > I agree to that. This is a valid concern but I do not think we have
> > many options. By our charter the project is intended to build re-
> > usable
> > components, not a single HTTP client.
> > 
> > Besides, we have somehow managed to keep HttpCore 4.x and
> > HttpClient
> > 4.x fully backward compatible for over 10 years, though it was not
> > easy.
> > 
> > Post 5.0 we should be able to upgrade to Java 1.8 or Java 9.
> > 
> > We can also decide that we can break API of classes / interfaces
> > marked
> > @Internal. I believe this is what Apache Lucene does.
> > 
> > In the meantime feel free to propose what classes / interfaces
> > should
> > be made package private or marked internal.
> > 
> > Oleg
> > 
> > 
> > 
> > 
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> > 
> > 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

Re: Core vs client

[Ryan Schmitt <rs...@apache.org>]

Point of clarification: what exactly does @Internal mean? Does it mean that
customers should not invoke methods on that type, or does it mean that they
should not import it at all? (In the latter case, we have more control,
since we can remove or merge types.)

On Wed, Dec 5, 2018 at 4:58 AM Oleg Kalnichevski <ol...@apache.org> wrote:

> On Tue, 2018-12-04 at 14:36 -0700, Gary Gregory wrote:
> > On Tue, Dec 4, 2018 at 12:54 PM Ryan Schmitt <rs...@apache.org>
> > wrote:
> >
> > > My concern with the core/client split is that I think it exposes
> > > tons of
> > > additional API surface area which will become a liability post-
> > > release. My
> > > recent changes for TLS handshake timeouts are a good example of
> > > this: in
> > > order to use the `connectTimeout` as the TLS handshake timeout
> > > (which is
> > > basically a bugfix), I had to make breaking API changes to over a
> > > dozen
> > > public classes, and two interfaces as well (TlsStrategy
> > > and TransportSecurityLayer).
> > >
> > > The interface changes are particularly worrisome, because after
> > > release
> > > they'll be more or less impossible; we won't even have Java 8
> > > default
> > > methods to fall back on. Core defines over 130 public interfaces,
> > > about
> > > half of which are new in 5.0, and only four of which are marked
> > > @Internal,
> > > so our compatibility obligations are clearly daunting.
> > >
> > > Merging core into client is obviously an extreme suggestion, but
> > > I'm not
> > > sure how else to attack the root of this problem.
> > >
> >
> > No silver bullet here I am afraid.
> >
>
> I agree to that. This is a valid concern but I do not think we have
> many options. By our charter the project is intended to build re-usable
> components, not a single HTTP client.
>
> Besides, we have somehow managed to keep HttpCore 4.x and HttpClient
> 4.x fully backward compatible for over 10 years, though it was not
> easy.
>
> Post 5.0 we should be able to upgrade to Java 1.8 or Java 9.
>
> We can also decide that we can break API of classes / interfaces marked
> @Internal. I believe this is what Apache Lucene does.
>
> In the meantime feel free to propose what classes / interfaces should
> be made package private or marked internal.
>
> Oleg
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>

Re: Core vs client

[Oleg Kalnichevski <ol...@apache.org>]

On Tue, 2018-12-04 at 14:36 -0700, Gary Gregory wrote:
> On Tue, Dec 4, 2018 at 12:54 PM Ryan Schmitt <rs...@apache.org>
> wrote:
> 
> > My concern with the core/client split is that I think it exposes
> > tons of
> > additional API surface area which will become a liability post-
> > release. My
> > recent changes for TLS handshake timeouts are a good example of
> > this: in
> > order to use the `connectTimeout` as the TLS handshake timeout
> > (which is
> > basically a bugfix), I had to make breaking API changes to over a
> > dozen
> > public classes, and two interfaces as well (TlsStrategy
> > and TransportSecurityLayer).
> > 
> > The interface changes are particularly worrisome, because after
> > release
> > they'll be more or less impossible; we won't even have Java 8
> > default
> > methods to fall back on. Core defines over 130 public interfaces,
> > about
> > half of which are new in 5.0, and only four of which are marked
> > @Internal,
> > so our compatibility obligations are clearly daunting.
> > 
> > Merging core into client is obviously an extreme suggestion, but
> > I'm not
> > sure how else to attack the root of this problem.
> > 
> 
> No silver bullet here I am afraid.
> 

I agree to that. This is a valid concern but I do not think we have
many options. By our charter the project is intended to build re-usable 
components, not a single HTTP client.

Besides, we have somehow managed to keep HttpCore 4.x and HttpClient
4.x fully backward compatible for over 10 years, though it was not
easy. 

Post 5.0 we should be able to upgrade to Java 1.8 or Java 9. 

We can also decide that we can break API of classes / interfaces marked
@Internal. I believe this is what Apache Lucene does.

In the meantime feel free to propose what classes / interfaces should
be made package private or marked internal.

Oleg




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

Re: Core vs client

[Gary Gregory <ga...@gmail.com>]

On Tue, Dec 4, 2018 at 12:54 PM Ryan Schmitt <rs...@apache.org> wrote:

> My concern with the core/client split is that I think it exposes tons of
> additional API surface area which will become a liability post-release. My
> recent changes for TLS handshake timeouts are a good example of this: in
> order to use the `connectTimeout` as the TLS handshake timeout (which is
> basically a bugfix), I had to make breaking API changes to over a dozen
> public classes, and two interfaces as well (TlsStrategy
> and TransportSecurityLayer).
>
> The interface changes are particularly worrisome, because after release
> they'll be more or less impossible; we won't even have Java 8 default
> methods to fall back on. Core defines over 130 public interfaces, about
> half of which are new in 5.0, and only four of which are marked @Internal,
> so our compatibility obligations are clearly daunting.
>
> Merging core into client is obviously an extreme suggestion, but I'm not
> sure how else to attack the root of this problem.
>

No silver bullet here I am afraid.

Documentation via Javadoc and annotations can help. The way Java works as
you know is that you must make APIs public in order for them to be visible
outside its package. It is quite likely that some APIs are public not for
user consumption but by mere necessity of making them accessible to other
classes in different packages. At that point, APIs are public as a
side-effect of organization of packages.

Even with documentation like @Internal, a public API is a public API, jar
hell is possible if you break BC. An alternative is to use '.internal.' in
package names but that still leaves the door open to jar hell.

I do not see anything else to do but code inspection and possibly moving
some code around to minimize public exposure. What is the extreme version
of this? If you put ALL the code in ONE package, you can only make public
what you really want, pretty ugly. Once you have all that you want public,
then maybe you can move code around back into packages... not fun.

The flip side is that we want things to be extendable where it makes sense
and that means public APIs and non-final classes and methods.

Another item to consider is should any classes and interfaces be final. You
loose sensibility of course but that also limits the openness of the
component.

Gary


> On Tue, Dec 4, 2018 at 3:21 AM Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Mon, 2018-12-03 at 14:11 -0800, Ryan Schmitt wrote:
> > > I had a few basic questions about the division between core and
> > > client:
> > >
> > > 1. Why the split between core and client?
> > >
> >
> > There are multiple reasons
> >
> > 1. Historic / internal ASF politics.
> > 2. Core has no mandatory external runtime dependencies (other than JRE)
> > while Client comes with a number of transitive dependencies: slf4j,
> > commons codec
> > 3. There are enough cases where core HTTP transport functionality is
> > perfectly enough, for instances, in reverse proxies or ESBs.
> >
> > > 2. Are there consumers of core who don't also consume client?
> > >
> >
> > I believe there are.
> >
> > > 3. What determines whether an abstraction lives in core or client?
> > > (For
> > > example,
> > >
> >
> > There is no clear-cut policy. Sometimes some things start off in Client
> > and later get moved to Core, if they turn out to be generic enough (for
> > example URIBuilder, SSLContextBuilder and connection pools)
> > Client side state management (cookies), various client side
> > authentication schemes, client connection routing, hostname
> > verification, publish suffix lists, etc on the other hand clearly have
> > no place in Core. Things that drag transitive dependencies with them do
> > not belong to Core.
> >
> >
> > > 4. Does core contain any interfaces that are intended for client use
> > > only?
> > >
> >
> > Depends how one defines client use only. HTTP proxies need certain
> > client side functionality, such as connection pooling.
> >
> > > 5. Why is there a dedicated set of client artifacts, but not server
> > > artifacts?
> > >
> >
> > Because we were mandated to de-emphasize server-side capabilities of
> > HttpComponents per our project charter. That mattered in year 2005 but
> > it no longer matters today. If we have enough people interested in
> > building a stand-alone HTTP server module we can push for revision of
> > the charter and start publishing purely server side artifacts.
> >
> > > 6. Will :httpcore5-h2 be merged into :httpcore5 before release? If
> > > not, is
> > > there a reason why it is being kept separate?
> >
> > What is the harm in keeping those modules separate?
> >
> > Oleg
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
> >
>

Re: Core vs client

[Ryan Schmitt <rs...@apache.org>]

My concern with the core/client split is that I think it exposes tons of
additional API surface area which will become a liability post-release. My
recent changes for TLS handshake timeouts are a good example of this: in
order to use the `connectTimeout` as the TLS handshake timeout (which is
basically a bugfix), I had to make breaking API changes to over a dozen
public classes, and two interfaces as well (TlsStrategy
and TransportSecurityLayer).

The interface changes are particularly worrisome, because after release
they'll be more or less impossible; we won't even have Java 8 default
methods to fall back on. Core defines over 130 public interfaces, about
half of which are new in 5.0, and only four of which are marked @Internal,
so our compatibility obligations are clearly daunting.

Merging core into client is obviously an extreme suggestion, but I'm not
sure how else to attack the root of this problem.

On Tue, Dec 4, 2018 at 3:21 AM Oleg Kalnichevski <ol...@apache.org> wrote:

> On Mon, 2018-12-03 at 14:11 -0800, Ryan Schmitt wrote:
> > I had a few basic questions about the division between core and
> > client:
> >
> > 1. Why the split between core and client?
> >
>
> There are multiple reasons
>
> 1. Historic / internal ASF politics.
> 2. Core has no mandatory external runtime dependencies (other than JRE)
> while Client comes with a number of transitive dependencies: slf4j,
> commons codec
> 3. There are enough cases where core HTTP transport functionality is
> perfectly enough, for instances, in reverse proxies or ESBs.
>
> > 2. Are there consumers of core who don't also consume client?
> >
>
> I believe there are.
>
> > 3. What determines whether an abstraction lives in core or client?
> > (For
> > example,
> >
>
> There is no clear-cut policy. Sometimes some things start off in Client
> and later get moved to Core, if they turn out to be generic enough (for
> example URIBuilder, SSLContextBuilder and connection pools)
> Client side state management (cookies), various client side
> authentication schemes, client connection routing, hostname
> verification, publish suffix lists, etc on the other hand clearly have
> no place in Core. Things that drag transitive dependencies with them do
> not belong to Core.
>
>
> > 4. Does core contain any interfaces that are intended for client use
> > only?
> >
>
> Depends how one defines client use only. HTTP proxies need certain
> client side functionality, such as connection pooling.
>
> > 5. Why is there a dedicated set of client artifacts, but not server
> > artifacts?
> >
>
> Because we were mandated to de-emphasize server-side capabilities of
> HttpComponents per our project charter. That mattered in year 2005 but
> it no longer matters today. If we have enough people interested in
> building a stand-alone HTTP server module we can push for revision of
> the charter and start publishing purely server side artifacts.
>
> > 6. Will :httpcore5-h2 be merged into :httpcore5 before release? If
> > not, is
> > there a reason why it is being kept separate?
>
> What is the harm in keeping those modules separate?
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>

Re: Core vs client

[Oleg Kalnichevski <ol...@apache.org>]

On Mon, 2018-12-03 at 14:11 -0800, Ryan Schmitt wrote:
> I had a few basic questions about the division between core and
> client:
> 
> 1. Why the split between core and client?
> 

There are multiple reasons

1. Historic / internal ASF politics.
2. Core has no mandatory external runtime dependencies (other than JRE)
while Client comes with a number of transitive dependencies: slf4j,
commons codec
3. There are enough cases where core HTTP transport functionality is
perfectly enough, for instances, in reverse proxies or ESBs.

> 2. Are there consumers of core who don't also consume client?
> 

I believe there are.

> 3. What determines whether an abstraction lives in core or client?
> (For
> example,
> 

There is no clear-cut policy. Sometimes some things start off in Client
and later get moved to Core, if they turn out to be generic enough (for
example URIBuilder, SSLContextBuilder and connection pools)
Client side state management (cookies), various client side
authentication schemes, client connection routing, hostname
verification, publish suffix lists, etc on the other hand clearly have
no place in Core. Things that drag transitive dependencies with them do
not belong to Core.


> 4. Does core contain any interfaces that are intended for client use
> only?
> 

Depends how one defines client use only. HTTP proxies need certain
client side functionality, such as connection pooling.

> 5. Why is there a dedicated set of client artifacts, but not server
> artifacts?
> 

Because we were mandated to de-emphasize server-side capabilities of
HttpComponents per our project charter. That mattered in year 2005 but
it no longer matters today. If we have enough people interested in
building a stand-alone HTTP server module we can push for revision of
the charter and start publishing purely server side artifacts.

> 6. Will :httpcore5-h2 be merged into :httpcore5 before release? If
> not, is
> there a reason why it is being kept separate?

What is the harm in keeping those modules separate?

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

Re: Core vs client

[Gary Gregory <ga...@gmail.com>]

My use case for HttpCore is a reverse proxy where HttpClient is only used
in tests.

Gary

On Mon, Dec 3, 2018, 15:12 Ryan Schmitt <rschmitt@apache.org wrote:

> I had a few basic questions about the division between core and client:
>
> 1. Why the split between core and client?
>
> 2. Are there consumers of core who don't also consume client?
>
> 3. What determines whether an abstraction lives in core or client? (For
> example,
>
> 4. Does core contain any interfaces that are intended for client use only?
>
> 5. Why is there a dedicated set of client artifacts, but not server
> artifacts?
>
> 6. Will :httpcore5-h2 be merged into :httpcore5 before release? If not, is
> there a reason why it is being kept separate?
>