More breaking changes in the Java API and how to deal with them

[Zoltan Ivanfi <zi...@cloudera.com>]

Hi,

In recent weeks several breaking changes have been discovered in minor
Parquet releases:

* https://issues.apache.org/jira/browse/PARQUET-1295
* https://issues.apache.org/jira/browse/PARQUET-1304
* https://issues.apache.org/jira/browse/PARQUET-1305

Parquet uses semantic versioning. As a library, it should take extra
care not to break its public API in minor releases. This also applies
to publicly accessible classes and methods that are considered
internal if this "internalness" is not properly documented. It is
tempting to dismiss these cases with the reasoning that they were not
intended to be public in the first place, but from an API consumer's
point of view, this "leaked" API is indistinguishable from the "real"
API. Currently the information of what is public and what is internal
is undocumented and only known to a few Parquet developers.

Until API consumers have a way to determine the intended target
audience of our classes and methods, we should pay more attention to
keeping our leaked internal API backwards-compatible as well. Gabor
posted some suggestions for doing so in
https://issues.apache.org/jira/browse/PARQUET-1295. Please join the
discussion there and let us know your opinions.

Thanks,

Zoltan

Re: More breaking changes in the Java API and how to deal with them

[Ryan Blue <rb...@netflix.com.INVALID>]

This is why I think we should define a parquet-api module and actually make
a proper API. Until we do that, we'll still have confusion over what is
public or not.

On Fri, May 18, 2018 at 9:25 AM, Zoltan Ivanfi <zi...@cloudera.com> wrote:

> Hi,
>
> In recent weeks several breaking changes have been discovered in minor
> Parquet releases:
>
> * https://issues.apache.org/jira/browse/PARQUET-1295
> * https://issues.apache.org/jira/browse/PARQUET-1304
> * https://issues.apache.org/jira/browse/PARQUET-1305
>
> Parquet uses semantic versioning. As a library, it should take extra
> care not to break its public API in minor releases. This also applies
> to publicly accessible classes and methods that are considered
> internal if this "internalness" is not properly documented. It is
> tempting to dismiss these cases with the reasoning that they were not
> intended to be public in the first place, but from an API consumer's
> point of view, this "leaked" API is indistinguishable from the "real"
> API. Currently the information of what is public and what is internal
> is undocumented and only known to a few Parquet developers.
>
> Until API consumers have a way to determine the intended target
> audience of our classes and methods, we should pay more attention to
> keeping our leaked internal API backwards-compatible as well. Gabor
> posted some suggestions for doing so in
> https://issues.apache.org/jira/browse/PARQUET-1295. Please join the
> discussion there and let us know your opinions.
>
> Thanks,
>
> Zoltan
>



-- 
Ryan Blue
Software Engineer
Netflix