You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Lin Sun <li...@gmail.com> on 2011/01/07 02:52:06 UTC
[COMPRESS]tar 2Gb limit?
Hi
While reading the user doc for the commons compress project -
http://commons.apache.org/compress/examples.html
Under tar section, it says -
The tar package does not support the full POSIX tar standard nor more
modern GNU extension of said standard. It cannot deal with entries
larger than 2 GByte either.
I wonder if this is still a limitation or should the limitation
changed to 8 GB instead of 2 GB. The reason I am asking is that I saw
a link that suggested someone complaining having probs with 10GB -
https://issues.apache.org/jira/browse/COMPRESS-16
Thanks much
Lin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [COMPRESS]tar 2Gb limit?
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-01-07, Lin Sun wrote:
> Do you know if this sentence is still valid -
> The tar package does not support the full POSIX tar standard nor more
> modern GNU extension of said standard
Absoluetly, yes.
> since you think compress project should support up to 8GB? I am
> hoping it is no longer valid. I don't really understand what it
> means. :-(
Unfortunately there is not one *tar* format.
The initial format used (ustar) before it was ever standardized by POSIX
is fully(?) supported. This format lacks quite a few important features
like support for file names longer than 100 characters, larger files and
more - so dialects emerged. The most popular variants stemmed from GNU
tar and PAX. Each of the dialects used the extension mechanism built
into to the original format but did so in different ways.
Commons Compress tar code detects some of the extensions when reading
archives and can write archives using the GNU extension for longer file
names. It does not support the full range of extensions, neither when
reading nor writing.
POSIX standardization came after the dialects emerged. It addressed
some of the original shortcommings, in part by using yet a different set
of extensions.
More recent versions of GNU tar (those less than four years old or so)
support the POSIX standard and will use it when creating new archives.
The "longfile" format used by Commons Compress is called "oldgnu" by GNU
tar and deprecated there.
> Also, is true streaming supported?
I'm not sure I understand the question. Tar archives consist of blocks
so sometimes it will be necessary to read more data than your read
requires (to read a full block) or writes will be delayed until a block
is full (or the archive is closed). In general it should be possible to
read/write archives in a streaming manner, though.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [COMPRESS]tar 2Gb limit?
Posted by Lin Sun <li...@gmail.com>.
Hi Stefan
Thanks much for the prompt reply. Good to know compress supports up
to 8GB and I am very eager to try it today. Do you know if this
sentence is still valid -
The tar package does not support the full POSIX tar standard nor more
modern GNU extension of said standard
since you think compress project should support up to 8GB? I am
hoping it is no longer valid. I don't really understand what it
means. :-(
Also, is true streaming supported?
Thanks again
Lin
On Fri, Jan 7, 2011 at 3:33 AM, Stefan Bodewig <bo...@apache.org> wrote:
> On 2011-01-07, Lin Sun wrote:
>
>> While reading the user doc for the commons compress project -
>> http://commons.apache.org/compress/examples.html
>
>> Under tar section, it says -
>
>> The tar package does not support the full POSIX tar standard nor more
>> modern GNU extension of said standard. It cannot deal with entries
>> larger than 2 GByte either.
>
>> I wonder if this is still a limitation or should the limitation
>> changed to 8 GB instead of 2 GB.
>
> I'm not sure anybody has ever confirmed success with an entry between 2
> and 8 GByte, so I don't really know. Looking at the code, you are most
> likely correct and the limit actually is 8 GByte.
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [COMPRESS]tar 2Gb limit?
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-01-07, Lin Sun wrote:
> While reading the user doc for the commons compress project -
> http://commons.apache.org/compress/examples.html
> Under tar section, it says -
> The tar package does not support the full POSIX tar standard nor more
> modern GNU extension of said standard. It cannot deal with entries
> larger than 2 GByte either.
> I wonder if this is still a limitation or should the limitation
> changed to 8 GB instead of 2 GB.
I'm not sure anybody has ever confirmed success with an entry between 2
and 8 GByte, so I don't really know. Looking at the code, you are most
likely correct and the limit actually is 8 GByte.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org