[compress][io] Avoid InputStream#skip?

[Stefan Bodewig <bo...@apache.org>]

Hi all

while looking into https://issues.apache.org/jira/browse/COMPRESS-449 I
realized that calling skip on any InputStream probably means you are up
to receiving an IOException because some stream in there decided to
implement skip via seek and now faces a stream that doesn't support
seek. And there is no way you can test for this.

It looks as if commons-io has decided to simple not use skip at all,
IOUtils.skip only uses read. commons-compress IOUtils.skip tries to be
smart with a combination of skip and read but is fooled when the stream
it reads from is System.in and the amount to skip is big enough.

OTOH not using skip but read on a stream that was happy to provide a
proper skip is a severe waste of time.

Any other ideas than playing save and wasting time?

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][io] Avoid InputStream#skip?

[Stefan Bodewig <bo...@apache.org>]

On 2018-04-23, sebb wrote:

> On 23 April 2018 at 19:45, Stefan Bodewig <bo...@apache.org> wrote:
>> On 2018-04-22, Gary Gregory wrote:

>>> We should at the very least document all of this carefully.

>> commons-io's IOUtils already quite clearly states it is using read and
>> not using skip at all.

>>> My gut reaction is that an API called skip should call skip internally.
>>> Doing anything else clever as you point out has performance implications
>>> and might violate the principle of least surprise.

>> Unfortunately I don't see any clean way to use skip at all.

> If skip only fails when trying to read a new buffer, it might be
> possible to get the file position before calling skip.
> This assumes that the failure does not occur after reading any further.

I've tried to implement something like this, see the COMPRESS-449
branch. Unfortunately it breaks the assumption that I can mark() the
stream before calling skip() and reset() afterwards - see
Pack200TestCase#testInputStreamMethods()

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][io] Avoid InputStream#skip?

[sebb <se...@gmail.com>]

On 23 April 2018 at 19:45, Stefan Bodewig <bo...@apache.org> wrote:
> On 2018-04-22, Gary Gregory wrote:
>
>> We should at the very least document all of this carefully.
>
> commons-io's IOUtils already quite clearly states it is using read and
> not using skip at all.
>
>> My gut reaction is that an API called skip should call skip internally.
>> Doing anything else clever as you point out has performance implications
>> and might violate the principle of least surprise.
>
> Unfortunately I don't see any clean way to use skip at all.

If skip only fails when trying to read a new buffer, it might be
possible to get the file position before calling skip.
This assumes that the failure does not occur after reading any further.

>> We could consider creating a Skipper interface that can be implemented by
>> with Skip and a SkipRead implementations. Then all of the code that used to
>> call skip would be parameterized. But this might be too framework-y for
>> [io], or [compress]
>
> Just grep for IOUtils.skip through the compress code base, it is used
> all over the place. :-)
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][io] Avoid InputStream#skip?

[Stefan Bodewig <bo...@apache.org>]

On 2018-04-22, Gary Gregory wrote:

> We should at the very least document all of this carefully.

commons-io's IOUtils already quite clearly states it is using read and
not using skip at all.

> My gut reaction is that an API called skip should call skip internally.
> Doing anything else clever as you point out has performance implications
> and might violate the principle of least surprise.

Unfortunately I don't see any clean way to use skip at all.

> We could consider creating a Skipper interface that can be implemented by
> with Skip and a SkipRead implementations. Then all of the code that used to
> call skip would be parameterized. But this might be too framework-y for
> [io], or [compress]

Just grep for IOUtils.skip through the compress code base, it is used
all over the place. :-)

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress][io] Avoid InputStream#skip?

[Gary Gregory <ga...@gmail.com>]

Hi All:

We should at the very least document all of this carefully.

My gut reaction is that an API called skip should call skip internally.
Doing anything else clever as you point out has performance implications
and might violate the principle of least surprise.

We could consider creating a Skipper interface that can be implemented by
with Skip and a SkipRead implementations. Then all of the code that used to
call skip would be parameterized. But this might be too framework-y for
[io], or [compress]

Gary

On Sun, Apr 22, 2018 at 9:52 AM, Stefan Bodewig <bo...@apache.org> wrote:

> Hi all
>
> while looking into https://issues.apache.org/jira/browse/COMPRESS-449 I
> realized that calling skip on any InputStream probably means you are up
> to receiving an IOException because some stream in there decided to
> implement skip via seek and now faces a stream that doesn't support
> seek. And there is no way you can test for this.
>
> It looks as if commons-io has decided to simple not use skip at all,
> IOUtils.skip only uses read. commons-compress IOUtils.skip tries to be
> smart with a combination of skip and read but is fooled when the stream
> it reads from is System.in and the amount to skip is big enough.
>
> OTOH not using skip but read on a stream that was happy to provide a
> proper skip is a severe waste of time.
>
> Any other ideas than playing save and wasting time?
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>