[compress] Dealing with "invalid" extra fields in ZIPs

[Stefan Bodewig <bo...@apache.org>]

Hi,

in ZIP files each entry can be associated with "extra data" for
extension stuff, this is available via ZipEntry#getExtra and
ZipArchiveEntry#getExtraField(s).

The semi-official ZIP standard recommends a format for this extra data
(two-byte tag, two-byte length of data, rest of data) and the current
ZIP classes in compress go a step further and enforce this
recommendation.

There are files out there that contain extra fields that do not follow
the recommendation.  You can find one attached to COMPRESS-62.

When reading such a file InfoZIP's unzip simply ignores the extra fields
and issues a warning.  Our ZipFile and ZipArchiveEntry classes throw an
exception.

It wouldn't be too difficult to make the codebase ignore those fields as
well, but there are a few things to consider:

* currently the Javadocs of ZipArchiveEntry#setExtra says "Throws an
  Exception if extra data cannot be parsed into extra fields."

  Either we change the Javadocs or we add a different method that does
  not throw an exceptions, I'm not sure.

* do we want to log a warning?  Where to?

* do we want to accept invalid extra fields when writing ZIP entries as
  well?

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress] Dealing with "invalid" extra fields in ZIPs

[Torsten Curdt <tc...@apache.org>]

>> So that's only a recommendation for a free form field?
>
> Yes, it's an array of arbitrary many bytes with a recommended internal
> structure.

I see

>>> * do we want to accept invalid extra fields when writing ZIP entries as
>>>  well?
>
>> If it really is just a free form field then we should accept anything.
>> Otherwise we should not encourage the creation of such files.
>
> I agree.  Just state the recommended format in the javadocs?

+1

cheers
--
Torsten

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress] Dealing with "invalid" extra fields in ZIPs

[Stefan Bodewig <bo...@apache.org>]

On 2010-02-12, Torsten Curdt <tc...@apache.org> wrote:

>> The semi-official ZIP standard recommends a format for this extra data
>> (two-byte tag, two-byte length of data, rest of data) and the current
>> ZIP classes in compress go a step further and enforce this
>> recommendation.

>> There are files out there that contain extra fields that do not follow
>> the recommendation.  You can find one attached to COMPRESS-62.

> So that's only a recommendation for a free form field?

Yes, it's an array of arbitrary many bytes with a recommended internal
structure.

>> When reading such a file InfoZIP's unzip simply ignores the extra fields
>> and issues a warning.  Our ZipFile and ZipArchiveEntry classes throw an
>> exception.

>> It wouldn't be too difficult to make the codebase ignore those fields as
>> well, but there are a few things to consider:

> Maybe we could introduce a boolean on whether we want to enforce this
> or not.
> Question is where this should be configured best and what the default
> behavior is.

The flag - if introduced - should be part of the stream classes and
ZipFile and apply to them as a whole.  When writing archives, you are
dealing with ZipArchiveEntry directly, though.

>> * do we want to log a warning?  Where to?

> Using logging is not really a great option for a library IMO.

Same here.

>> * do we want to accept invalid extra fields when writing ZIP entries as
>>  well?

> If it really is just a free form field then we should accept anything.
> Otherwise we should not encourage the creation of such files.

I agree.  Just state the recommended format in the javadocs?

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [compress] Dealing with "invalid" extra fields in ZIPs

[Torsten Curdt <tc...@apache.org>]

> The semi-official ZIP standard recommends a format for this extra data
> (two-byte tag, two-byte length of data, rest of data) and the current
> ZIP classes in compress go a step further and enforce this
> recommendation.
>
> There are files out there that contain extra fields that do not follow
> the recommendation.  You can find one attached to COMPRESS-62.

So that's only a recommendation for a free form field?

> When reading such a file InfoZIP's unzip simply ignores the extra fields
> and issues a warning.  Our ZipFile and ZipArchiveEntry classes throw an
> exception.
>
> It wouldn't be too difficult to make the codebase ignore those fields as
> well, but there are a few things to consider:

Maybe we could introduce a boolean on whether we want to enforce this or not.
Question is where this should be configured best and what the default
behavior is.

> * currently the Javadocs of ZipArchiveEntry#setExtra says "Throws an
>  Exception if extra data cannot be parsed into extra fields."
>
>  Either we change the Javadocs or we add a different method that does
>  not throw an exceptions, I'm not sure.
>
> * do we want to log a warning?  Where to?

Using logging is not really a great option for a library IMO.

> * do we want to accept invalid extra fields when writing ZIP entries as
>  well?

If it really is just a free form field then we should accept anything.
Otherwise we should not encourage the creation of such files.

IMO

cheers
--
Torsten

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org