You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "jessy (JIRA)" <ji...@apache.org> on 2012/07/06 10:49:36 UTC
[jira] [Created] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
jessy created COMPRESS-194:
------------------------------
Summary: Unable to create a TAR file that contains an entry which is >8 GB in size.
Key: COMPRESS-194
URL: https://issues.apache.org/jira/browse/COMPRESS-194
Project: Commons Compress
Issue Type: Bug
Components: Archivers
Affects Versions: 1.4.1
Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
Reporter: jessy
Priority: Critical
The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
Only Untar operations on a file > 8GB is fixed,
But creating a tar on a file >8GB is not working.
When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
i get the below exception.
java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (COMPRESS-194) Unable to create a TAR file
that contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410055#comment-13410055 ]
jessy edited comment on COMPRESS-194 at 7/10/12 5:15 AM:
---------------------------------------------------------
Stefan,
Thank You for your timely response.
I ported the tar archive to linux box and did untarring. It looks all good with no loss of data. Have used STAR .
However, winrar in windows does not support extracting tar file of >8GB . Had to extract using 7-zip to extract successfully to view the contents.
:-)
was (Author: jessy_2012):
Stefan,
I ported the tar archive to linux box and did untarring. It looks all good with no loss of data. Have used STAR .
However, winrar in windows does not support extracting tar file of >8GB . Had to extract using 7-zip to extract successfully to view the contents.
:-)
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409297#comment-13409297 ]
jessy commented on COMPRESS-194:
--------------------------------
Stefan,
I tried to setBigNumberMode to tarOutputStream.BIGNUMBER_STAR [ =1 ].
However, i found that tarring completes without any exceptions, and shows the tar archive size as 9GB(which is my input of 9GB content file).
But ,when i tried to unzip( windows ), I was unsuccessful in opening this tar. Got the error dialog "The archive is corrupt”
When I tried to open the tar thru windows right click menu , I found my content file size is 0 bytes. I have taken care to close all streams .
Could you kindly look at my below code snippet and suggest what might be the rootcause for this.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
import org.apache.log4j.Logger;
/**
* The Class TarArchiver.
*/
public class TarArchiver {
/** The Constant TWO_KILO_BYTES. */
private static final int TWO_KILO_BYTES = 2048;
/** The tar output stream. */
private TarArchiveOutputStream tarOutputStream;
/** The file output stream. */
private FileOutputStream fileOutputStream;
/** The Constant EOF. */
private static final int EOF = -1;
/** The Constant LOGGER. */
private static final Logger LOGGER = Logger.getLogger(TarArchiver.class);
/**
* Instantiates a new tar archiver.
*
* @param directory the directory
* @param tarName the tar name
* @throws IOException Signals that an I/O exception has occurred.
* @throws CustomException the custom exception
*/
public void archive(String directory, String tarName) throws IOException, CustomException {
tarOutputStream = new TarArchiveOutputStream(new FileOutputStream(tarName));
tarOutputStream.setBigNumberMode(tarOutputStream.BIGNUMBER_STAR);
String path = "";
try {
archiveDir(directory, path);
}
finally {
tarOutputStream.close();
}
}
/**
* Archive dir.
*
* @param directory the directory
* @param path the path
* @throws CustomException the custom exception
*/
private void archiveDir(String directory, String path) throws CustomException {
byte[] dataBuffer = new byte[TWO_KILO_BYTES];
fileOutputStream = null;
File tarDir = null;
try {
tarDir = new File(directory);
String files[] = tarDir.list();
for (int count = 0; count < files.length; count++) {
File f = new File(tarDir, files[count]);
createTarArchive(dataBuffer, path, f);
}
tarOutputStream.flush();
}
catch (FileNotFoundException e) {
LOGGER.error("FileNotFoundException to archive : " + tarDir, e);
throw new CustomException(e.getMessage(), e);
}
catch (IOException e) {
LOGGER.error("IOException during archiving : " + tarDir, e);
throw new CustomException(e.getMessage(), e);
}
}
/**
* Creates the tar archive.
*
* @param dataBuffer the data buffer
* @param path the path
* @param file the file
* @throws IOException Signals that an I/O exception has occurred.
*/
private void createTarArchive(byte[] dataBuffer, String path, File file) throws IOException {
{
FileInputStream fileInputStream = new FileInputStream(file);
try {
int bytesIn = 0;
String string = path + file.getName();
TarArchiveEntry archiveEntry = new TarArchiveEntry(file, string);
tarOutputStream.putArchiveEntry(archiveEntry);
bytesIn = fileInputStream.read(dataBuffer);
while (bytesIn != EOF) {
tarOutputStream.write(dataBuffer, 0, bytesIn);
bytesIn = fileInputStream.read(dataBuffer);
}
tarOutputStream.closeArchiveEntry();
}
catch (RuntimeException ex) {
closeInputStream(fileInputStream);
closeOutputStream();
}
finally {
closeInputStream(fileInputStream);
}
}
}
private void closeOutputStream() {
closeOutputStream(fileOutputStream);
closeOutputStream(tarOutputStream);
}
private void closeOutputStream(OutputStream os) {
try {
if (os != null) {
os.close();
}
}
catch (IOException e) {
LOGGER.error("Failed to close Output Stream.", e);
}
}
private void closeInputStream(InputStream is) {
try {
if (is != null) {
is.close();
}
}
catch (IOException e) {
LOGGER.error("Failed to close Input Stream.", e);
}
}
Jes
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (COMPRESS-194) Unable to create a TAR file
that contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409300#comment-13409300 ]
jessy edited comment on COMPRESS-194 at 7/9/12 10:47 AM:
---------------------------------------------------------
Stefan,
I tried to setBigNumberMode to tarOutputStream.BIGNUMBER_STAR [ =1 ].
However, i found that tarring completes without any exceptions, and shows the tar archive size as 9GB(which is my input of 9GB content file).
But ,when i tried to unzip using winrar( windows ), I was unsuccessful in opening this tar. Got the error dialog "The archive is corrupt”
When I tried to open the tar thru windows right click menu , I found my content file size is 0 bytes. I have taken care to close all streams .
I have attached my code snippet with this thread.
Could you kindly look at my below code snippet and suggest what might be the rootcause for this.
Regards,
Jes
was (Author: jessy_2012):
Stefan,
I tried to setBigNumberMode to tarOutputStream.BIGNUMBER_STAR [ =1 ].
However, i found that tarring completes without any exceptions, and shows the tar archive size as 9GB(which is my input of 9GB content file).
But ,when i tried to unzip( windows ), I was unsuccessful in opening this tar. Got the error dialog "The archive is corrupt”
When I tried to open the tar thru windows right click menu , I found my content file size is 0 bytes. I have taken care to close all streams .
I have attached my code snippet with this thread.
Could you kindly look at my below code snippet and suggest what might be the rootcause for this.
Regards,
Jes
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "Stefan Bodewig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410045#comment-13410045 ]
Stefan Bodewig commented on COMPRESS-194:
-----------------------------------------
I don't know which dialect of tar Winrar supports, but I'd rather expect it to support POSIX/PAX than STAR. In fact it is possible it doesn't support either. The same is true for right-clicking on Windows (I didn't know it would support tar at all).
If I was you I'd try POSIX and if that fails research if either of your targets supports tars with entries bigger than 8GB at all - and if so which dialect they want to speak.
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409300#comment-13409300 ]
jessy commented on COMPRESS-194:
--------------------------------
Stefan,
I tried to setBigNumberMode to tarOutputStream.BIGNUMBER_STAR [ =1 ].
However, i found that tarring completes without any exceptions, and shows the tar archive size as 9GB(which is my input of 9GB content file).
But ,when i tried to unzip( windows ), I was unsuccessful in opening this tar. Got the error dialog "The archive is corrupt”
When I tried to open the tar thru windows right click menu , I found my content file size is 0 bytes. I have taken care to close all streams .
I have attached my code snippet with this thread.
Could you kindly look at my below code snippet and suggest what might be the rootcause for this.
Regards,
Jes
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410055#comment-13410055 ]
jessy commented on COMPRESS-194:
--------------------------------
Stefan,
I ported the tar archive to linux box and did untarring. It looks all good with no loss of data. Have used STAR .
However, winrar in windows does not support extracting tar file of >8GB . Had to extract using 7-zip to extract successfully to view the contents.
:-)
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jessy updated COMPRESS-194:
---------------------------
Attachment: tarImplntion.txt
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "Stefan Bodewig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stefan Bodewig resolved COMPRESS-194.
-------------------------------------
Resolution: Not A Problem
Fix Version/s: 1.4
The docs of Ant's compress Antlib are not exactly Commons Compress' docs ;-)
If you want to create archives with big files you have to explicitly enable it by setting bigNumberMode to something other than TarArchiveOutputStream.BIGNUMBER_ERROR (which is the default), i.e. you have to chose between posix and star modes.
In the case of the Compress Antlib it means you have to chose a format that supports big files in the tar task, i.e. "star", "gnu" or "posix".
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jessy updated COMPRESS-194:
---------------------------
Comment: was deleted
(was: Stefan,
I tried to setBigNumberMode to tarOutputStream.BIGNUMBER_STAR [ =1 ].
However, i found that tarring completes without any exceptions, and shows the tar archive size as 9GB(which is my input of 9GB content file).
But ,when i tried to unzip( windows ), I was unsuccessful in opening this tar. Got the error dialog "The archive is corrupt”
When I tried to open the tar thru windows right click menu , I found my content file size is 0 bytes. I have taken care to close all streams .
Could you kindly look at my below code snippet and suggest what might be the rootcause for this.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
import org.apache.log4j.Logger;
/**
* The Class TarArchiver.
*/
public class TarArchiver {
/** The Constant TWO_KILO_BYTES. */
private static final int TWO_KILO_BYTES = 2048;
/** The tar output stream. */
private TarArchiveOutputStream tarOutputStream;
/** The file output stream. */
private FileOutputStream fileOutputStream;
/** The Constant EOF. */
private static final int EOF = -1;
/** The Constant LOGGER. */
private static final Logger LOGGER = Logger.getLogger(TarArchiver.class);
/**
* Instantiates a new tar archiver.
*
* @param directory the directory
* @param tarName the tar name
* @throws IOException Signals that an I/O exception has occurred.
* @throws CustomException the custom exception
*/
public void archive(String directory, String tarName) throws IOException, CustomException {
tarOutputStream = new TarArchiveOutputStream(new FileOutputStream(tarName));
tarOutputStream.setBigNumberMode(tarOutputStream.BIGNUMBER_STAR);
String path = "";
try {
archiveDir(directory, path);
}
finally {
tarOutputStream.close();
}
}
/**
* Archive dir.
*
* @param directory the directory
* @param path the path
* @throws CustomException the custom exception
*/
private void archiveDir(String directory, String path) throws CustomException {
byte[] dataBuffer = new byte[TWO_KILO_BYTES];
fileOutputStream = null;
File tarDir = null;
try {
tarDir = new File(directory);
String files[] = tarDir.list();
for (int count = 0; count < files.length; count++) {
File f = new File(tarDir, files[count]);
createTarArchive(dataBuffer, path, f);
}
tarOutputStream.flush();
}
catch (FileNotFoundException e) {
LOGGER.error("FileNotFoundException to archive : " + tarDir, e);
throw new CustomException(e.getMessage(), e);
}
catch (IOException e) {
LOGGER.error("IOException during archiving : " + tarDir, e);
throw new CustomException(e.getMessage(), e);
}
}
/**
* Creates the tar archive.
*
* @param dataBuffer the data buffer
* @param path the path
* @param file the file
* @throws IOException Signals that an I/O exception has occurred.
*/
private void createTarArchive(byte[] dataBuffer, String path, File file) throws IOException {
{
FileInputStream fileInputStream = new FileInputStream(file);
try {
int bytesIn = 0;
String string = path + file.getName();
TarArchiveEntry archiveEntry = new TarArchiveEntry(file, string);
tarOutputStream.putArchiveEntry(archiveEntry);
bytesIn = fileInputStream.read(dataBuffer);
while (bytesIn != EOF) {
tarOutputStream.write(dataBuffer, 0, bytesIn);
bytesIn = fileInputStream.read(dataBuffer);
}
tarOutputStream.closeArchiveEntry();
}
catch (RuntimeException ex) {
closeInputStream(fileInputStream);
closeOutputStream();
}
finally {
closeInputStream(fileInputStream);
}
}
}
private void closeOutputStream() {
closeOutputStream(fileOutputStream);
closeOutputStream(tarOutputStream);
}
private void closeOutputStream(OutputStream os) {
try {
if (os != null) {
os.close();
}
}
catch (IOException e) {
LOGGER.error("Failed to close Output Stream.", e);
}
}
private void closeInputStream(InputStream is) {
try {
if (is != null) {
is.close();
}
}
catch (IOException e) {
LOGGER.error("Failed to close Input Stream.", e);
}
}
Jes
)
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COMPRESS-194) Unable to create a TAR file that
contains an entry which is >8 GB in size.
Posted by "Jessy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COMPRESS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jessy closed COMPRESS-194.
--------------------------
Well Done :-)
> Unable to create a TAR file that contains an entry which is >8 GB in size.
> ---------------------------------------------------------------------------
>
> Key: COMPRESS-194
> URL: https://issues.apache.org/jira/browse/COMPRESS-194
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
> Affects Versions: 1.4.1
> Environment: I am using win xp and Red HatLinux 5.1v, but this should be platform independent.
> Reporter: Jessy
> Priority: Critical
> Fix For: 1.4
>
> Attachments: tarImplntion.txt
>
>
> The common-compress-1.4.1 api says it supports unlimited file sizes for tar and untar operations. [http://ant.apache.org/antlibs/compress/]
> Only Untar operations on a file > 8GB is fixed,
> But creating a tar on a file >8GB is not working.
> When I try to do " tarOutputStream.putArchiveEntry(archiveEntry); "
> i get the below exception.
> java.lang.RuntimeException: entry size '9633985364' is too big ( > 8589934591 )
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:572)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:557)
> at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:297)
> Looking for a fix ..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira