[compress][POLL] High Level API

[Stefan Bodewig <bo...@apache.org>]

Hi all

right now the master branch contains two different ideas of hign level
APIs:

* https://github.com/apache/commons-compress/blob/master/src/main/java/org/apache/commons/compress/archivers/Archiver.java
  and Expander in the same package

* https://github.com/apache/commons-compress/tree/master/src/main/java/org/apache/commons/compress/archivers/examples

Neither of which has any unit tests. As writing the tests will take some
time I'd like to avoid writing them for the wrong API :-)

So I'd like to get a feeling for where we want to go:

* drop both (I can live with that) as we don't want to provide a high
  level API at all - in that case I'd close COMPRESS-118 as WONTFIX

* drop both as neither looks good (I can live with that, too)

* keep the two classes with static methods - and maybe move them to an
  examples package, remove the existing content from the examples
  package

* keep the examples package remove the two classes with static methods

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][POLL] High Level API

[Gary Gregory <ga...@gmail.com>]

On Wed, May 2, 2018 at 2:13 PM, Stefan Bodewig <bo...@apache.org> wrote:

> On 2018-05-02, Torsten Curdt wrote:
>
> > So far I didn't think the current API is so horrible.
>
> I wouldn't call it horrible.
>
> My ideas about things that should be different and have been epressed in
> the compress-2.0 branch I started years ago. To me the most important
> things I'd want to change are
>
> * have self-describing formats like "supports random access", "is
>   read-only", "is it possible to detect the format by looking into the
>   first few bytes.
>
> * have a unified interface for stream like and random access modes
>
> * separate the archive from the stream. It doesn't feel right to read
>   from an input stream until it returns -1, then advance some kind o
>   stream cursor and read from the same stream again.
>
> * have a more unified view on archive entries (file modes, ownership
>   information and similar things that tar/cpio/ar and zip provide in
>   slightly different ways)
>
> Neither of these points is related to a high level API, though.
>
> But people ask for a higher level API for "create a ZIP from that
> directory" or "extract that archive to the directory over there". Lack
> of a "one line API" is something I've seen as criticism on StackOverflow
> frequently, but maybe I shouldn't hang out there :-)
>

I think you can do all that with Commons VFS.

Gary


>
> > Maybe it's rather worth going into a possible unreleased 2.0 branch?
>
> Tried that, didn't work too well for my past experiments.
>
> >>> I personally would keep both approaches for now - but somewhere
> >>> outside of the official jar.
>
> As this drags on I think it would be best to just spin the code off to
> another branch and get master back into a state we'd feel comforable
> with and consider releaseable.
>
> It's probably better to give the discussion some more time.
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Re: [compress][POLL] High Level API

[Stefan Bodewig <bo...@apache.org>]

On 2018-05-02, Torsten Curdt wrote:

> So far I didn't think the current API is so horrible.

I wouldn't call it horrible.

My ideas about things that should be different and have been epressed in
the compress-2.0 branch I started years ago. To me the most important
things I'd want to change are

* have self-describing formats like "supports random access", "is
  read-only", "is it possible to detect the format by looking into the
  first few bytes.

* have a unified interface for stream like and random access modes

* separate the archive from the stream. It doesn't feel right to read
  from an input stream until it returns -1, then advance some kind o
  stream cursor and read from the same stream again.

* have a more unified view on archive entries (file modes, ownership
  information and similar things that tar/cpio/ar and zip provide in
  slightly different ways)

Neither of these points is related to a high level API, though.

But people ask for a higher level API for "create a ZIP from that
directory" or "extract that archive to the directory over there". Lack
of a "one line API" is something I've seen as criticism on StackOverflow
frequently, but maybe I shouldn't hang out there :-)

> Maybe it's rather worth going into a possible unreleased 2.0 branch?

Tried that, didn't work too well for my past experiments.

>>> I personally would keep both approaches for now - but somewhere
>>> outside of the official jar.

As this drags on I think it would be best to just spin the code off to
another branch and get master back into a state we'd feel comforable
with and consider releaseable.

It's probably better to give the discussion some more time.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][POLL] High Level API

[Torsten Curdt <tc...@vafer.org>]

>
> > Given that you are currently the main person working on Compress I'd say
> -
> > whatever you are OK with.
> > But you don't really sound super confident/happy about the API -
> > otherwise you might not have written this email :)
>
> TBH I've written this email because my compass for which direction
> Compress want to go seems to be severly off. When I started the initial
> discussion about how a high level API might look I didn't expect that
> those who responded would say "we don't want to maintain a high level
> API at all".
>
> Don't get me wrong, I'm not compaining, not at all, just completely
> unsure what the community actually wants.


...and that's actually the main (if not only) reason why I am so hesitant
to give a +1 for adding it as is.

So far I didn't think the current API is so horrible.
But if I (as a user) don't see a benefit in switching
then I am wondering about other peoples use cases.
...and whether it's worth adding to maintain.

Maybe it's rather worth going into a possible unreleased 2.0 branch?
Rethinking the API?


.If this only was about example code then I'd be perfectly happy with the
> smaller initial idea and probably would even strip out the filtering
> parts in order to reduce the number of overloads by half. If we wanted
> to provide a useful high level API I'd prefer the second version.
>
> > I personally would keep both approaches for now - but somewhere
> > outside of the official jar.  And just give everyone some time to play
> > with it.
>
> Which is another twist I didn't expect. Don't ship the example/high
> level stuff with the main artifact at all. Which is also fine with me,
> as long as it gets compiled and tested whenever we change "the real
> library".
>

Well, or use an experimental package. Just not a great fan of those.
It's like showing users a big red button telling them not to press it.

On the other hand - how we get good feedback? It's not like it's a big
project with many user on the users list.
Right now I only see the option to check maven central for projects that
use compress and look at their code.

Maybe we get some feedback when they press the red button ;)

cheers,
Torsten

Re: [compress][POLL] High Level API

[Stefan Bodewig <bo...@apache.org>]

On 2018-05-02, Gary Gregory wrote:

> I'm all for providing a high-level API. That's fine.

> I would like a high-level statement first though concerning choices we have
> or have not considered.

> - The high level API is Commons VFS. Why? Why not?
> - The high level API is Java IO File System.  Why? Why not?

Personal opinion: I've never been involved with Commons VFS but was
under the impression it already contained a backend for compress. And if
VFS was the high leel API it would feel more natural for its
implementation to live in VFS as well.

File system has never occured to me as an abstraction for things I do to
an archive, that may have been short-sighted.

Either of these options would rule out anything for the 1.17 release but
there is no need to rush anything.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][POLL] High Level API

[Matt Sicker <bo...@gmail.com>]

On 2 May 2018 at 09:01, Gary Gregory <ga...@gmail.com> wrote:

> - The high level API is Commons VFS. Why? Why not?
> - The high level API is Java IO File System.  Why? Why not?
>

This really gets at what I've been thinking. Compress and VFS are strongly
related, and both could likely benefit from adopting the Java NIO2 API.
That doesn't even begin to touch things like using off-heap byte buffers
and native codec libraries and such (e.g., using the unsafe variant of XZ's
API), nor does it touch async file APIs and other goodies.

-- 
Matt Sicker <bo...@gmail.com>

Re: [compress][POLL] High Level API

[Gary Gregory <ga...@gmail.com>]

I'm all for providing a high-level API. That's fine.

I would like a high-level statement first though concerning choices we have
or have not considered.

- The high level API is Commons VFS. Why? Why not?
- The high level API is Java IO File System.  Why? Why not?

Gary

On Wed, May 2, 2018 at 7:51 AM, Stefan Bodewig <bo...@apache.org> wrote:

> On 2018-05-01, Torsten Curdt wrote:
>
> >  https://github.com/apache/commons-compress/blob/master/
> > src/main/java/org/apache/commons/compress/archivers/Archiver.java
>
> > Is tiny compared the whole lots of
>
> > https://github.com/apache/commons-compress/tree/master/
> > src/main/java/org/apache/commons/compress/archivers/examples
>
> > and while the later is not a bad approach it's also a whole lot of belts
> > and whistles.
>
> True.
>
> > Bottom line:
>
> > Given that you are currently the main person working on Compress I'd say
> -
> > whatever you are OK with.
> > But you don't really sound super confident/happy about the API -
> > otherwise you might not have written this email :)
>
> TBH I've written this email because my compass for which direction
> Compress want to go seems to be severly off. When I started the initial
> discussion about how a high level API might look I didn't expect that
> those who responded would say "we don't want to maintain a high level
> API at all".
>
> Don't get me wrong, I'm not compaining, not at all, just completely
> unsure what the community actually wants. And the last thing I want to
> do is to impose my ideas onto a group of people who don't want them but
> let me have my way because I happen to be the one doing some work right
> now.
>
> If this only was about example code then I'd be perfectly happy with the
> smaller initial idea and probably would even strip out the filtering
> parts in order to reduce the number of overloads by half. If we wanted
> to provide a useful high level API I'd prefer the second version.
>
> > I personally would keep both approaches for now - but somewhere
> > outside of the official jar.  And just give everyone some time to play
> > with it.
>
> Which is another twist I didn't expect. Don't ship the example/high
> level stuff with the main artifact at all. Which is also fine with me,
> as long as it gets compiled and tested whenever we change "the real
> library".
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Re: [compress][POLL] High Level API

[Stefan Bodewig <bo...@apache.org>]

On 2018-05-01, Torsten Curdt wrote:

>  https://github.com/apache/commons-compress/blob/master/
> src/main/java/org/apache/commons/compress/archivers/Archiver.java

> Is tiny compared the whole lots of

> https://github.com/apache/commons-compress/tree/master/
> src/main/java/org/apache/commons/compress/archivers/examples

> and while the later is not a bad approach it's also a whole lot of belts
> and whistles.

True.

> Bottom line:

> Given that you are currently the main person working on Compress I'd say -
> whatever you are OK with.
> But you don't really sound super confident/happy about the API -
> otherwise you might not have written this email :)

TBH I've written this email because my compass for which direction
Compress want to go seems to be severly off. When I started the initial
discussion about how a high level API might look I didn't expect that
those who responded would say "we don't want to maintain a high level
API at all".

Don't get me wrong, I'm not compaining, not at all, just completely
unsure what the community actually wants. And the last thing I want to
do is to impose my ideas onto a group of people who don't want them but
let me have my way because I happen to be the one doing some work right
now.

If this only was about example code then I'd be perfectly happy with the
smaller initial idea and probably would even strip out the filtering
parts in order to reduce the number of overloads by half. If we wanted
to provide a useful high level API I'd prefer the second version.

> I personally would keep both approaches for now - but somewhere
> outside of the official jar.  And just give everyone some time to play
> with it.

Which is another twist I didn't expect. Don't ship the example/high
level stuff with the main artifact at all. Which is also fine with me,
as long as it gets compiled and tested whenever we change "the real
library".

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][POLL] High Level API

[Torsten Curdt <tc...@vafer.org>]

 https://github.com/apache/commons-compress/blob/master/
src/main/java/org/apache/commons/compress/archivers/Archiver.java

Is tiny compared the whole lots of

https://github.com/apache/commons-compress/tree/master/
src/main/java/org/apache/commons/compress/archivers/examples

and while the later is not a bad approach it's also a whole lot of belts
and whistles.

I also noticed when re-reading the snippet

    try (Sink<File> sink = ArchiveSinks.forFile(args[1], new
File(args[2]))) {
        Archive.directory(new File(args[0]))
            .to(sink);
    }

that "forFile" is not very descriptive for a high level API.


With jdeb being one of the compress users I am still on the fence whether
(if released some day) I would want to switch to a Sink API or not.
Internally jdeb also uses producer-consumer abstraction - so it could maybe
make sense to try that route.
And with jdeb needing a major overhaul anyway it is a possibility.
But I also have the feeling I might want to have the producer-consumer code
closer to home.
Either way I'd need to find the time to sketch out the plans for jdeb 2.x
first.

Bottom line:

Given that you are currently the main person working on Compress I'd say -
whatever you are OK with.
But you don't really sound super confident/happy about the API - otherwise
you might not have written this email :)

I personally would keep both approaches for now - but somewhere outside of
the official jar.
And just give everyone some time to play with it.

cheers,
Torsten