You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by gg...@apache.org on 2025/08/06 12:37:49 UTC
(commons-compress) branch master updated: org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory
This is an automated email from the ASF dual-hosted git repository.
ggregory pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-compress.git
The following commit(s) were added to refs/heads/master by this push:
new d59a49ff8 org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory
d59a49ff8 is described below
commit d59a49ff8b47f25462caa8f20d18472120b4e9c7
Author: Gary D. Gregory <ga...@gmail.com>
AuthorDate: Wed Aug 6 08:37:43 2025 -0400
org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry()
now throws a MemoryLimitException instead of OutOfMemoryError when it
can't process input greater than available memory
- Add org.apache.commons.compress.archivers.ArchiveException.toIntExact(long)
---
src/changes/changes.xml | 6 ++++--
.../apache/commons/compress/MemoryLimitException.java | 13 +++++++++++--
.../commons/compress/archivers/ArchiveException.java | 16 ++++++++++++++++
.../compress/archivers/cpio/CpioArchiveInputStream.java | 2 +-
.../java/org/apache/commons/compress/utils/IOUtils.java | 14 ++++++++++----
.../archivers/cpio/CpioArchiveInputStreamTest.java | 7 ++++---
6 files changed, 46 insertions(+), 12 deletions(-)
diff --git a/src/changes/changes.xml b/src/changes/changes.xml
index abf242be9..433f3b5b0 100644
--- a/src/changes/changes.xml
+++ b/src/changes/changes.xml
@@ -54,7 +54,7 @@ The <action> type attribute can be add,update,fix,remove.
<action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">Fix format string in org.apache.commons.compress.archivers.sevenz.SevenZFile.assertFitsIntoNonNegativeInt(String, long).</action>
<action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.sevenz.SevenZFile.Builder.get() now throws a MemoryLimitException (an IOException subclass) instead of an OOME.</action>
<action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream constructors now throws CompressorException instead of ArrayIndexOutOfBoundsException.</action>
- <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.TapeInputStream.resetBlockSize(int, boolean) now throws a MemoryLimitException (an IOException subclass) instead of an OOME.</action>
+ <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.TapeInputStream.resetBlockSize(int, boolean) now throws a MemoryLimitException (an IOException subclass) instead of an OutOfMemoryError.</action>
<action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.DumpArchiveInputStream.getNextEntry() throws an ArchiveException instead of ArrayIndexOutOfBoundsException.</action>
<action type="fix" dev="ggregory" due-to="Dominik Stadler, Gary Gregory" issue="COMPRESS-598">org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(byte[], int, int) now throws an IOException instead of a NullPointerException.</action>
<action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.zip.ZipFile.createBoundedInputStream(long, long) now throws an ArchiveException instead of IllegalArgumentException.</action>
@@ -72,7 +72,8 @@ The <action> type attribute can be add,update,fix,remove.
<action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.archivers.zip.ExplodingInputStream now throws ArchiveException instead of Arithmetic exception.</action>
<action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.harmony.pack200.RunCodec now throws ArchiveException instead of Arithmetic exception.</action>
<action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.harmony.pack200.FileBands now throws ArchiveException instead of Arithmetic exception.</action>
- <action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveEntry now throws ArchiveException instead of Arithmetic exception.</action>
+ <action type="fix" dev="ggregory" due-to="Stan, Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveEntry now throws ArchiveException instead of Arithmetic exception.</action>
+ <action type="fix" dev="ggregory" due-to="Stan, Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory.</action>
<!-- ADD -->
<action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.MemoryLimitException.MemoryLimitException(long, long).</action>
<action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.CompressException.CompressException(String, Object...).</action>
@@ -84,6 +85,7 @@ The <action> type attribute can be add,update,fix,remove.
<action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(int, int).</action>
<action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(int, long).</action>
<action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(long, long).</action>
+ <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.toIntExact(long).</action>
<!-- UPDATE -->
</release>
<release version="1.28.0" date="2025-07-26" description="This is a feature and maintenance release. Java 8 or later is required.">
diff --git a/src/main/java/org/apache/commons/compress/MemoryLimitException.java b/src/main/java/org/apache/commons/compress/MemoryLimitException.java
index 1fbf7f3c1..31614100b 100644
--- a/src/main/java/org/apache/commons/compress/MemoryLimitException.java
+++ b/src/main/java/org/apache/commons/compress/MemoryLimitException.java
@@ -19,6 +19,8 @@
package org.apache.commons.compress;
+import java.io.ByteArrayOutputStream;
+
/**
* If a stream checks for estimated memory allocation, and the estimate goes above the memory limit, this is thrown. This can also be thrown if a stream tries
* to allocate a byte array that is larger than the allowable limit.
@@ -27,17 +29,24 @@
*/
public class MemoryLimitException extends CompressException {
+ /**
+ * The maximum array size defined privately in {@link ByteArrayOutputStream}.
+ *
+ * @since 1.29.0
+ */
+ public static int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
+
private static final String MAXIMUM = "maximum";
private static final String BYTES = "bytes";
private static final String KIB = "KiB";
private static final long serialVersionUID = 1L;
private static String buildMessage(final long memoryNeeded, final long memoryLimit, final String scale, final String boundary) {
- return String.format("%,d %s of memory would be needed; %s was %,d %s. If the file is not corrupt, consider increasing the memory limit.", memoryNeeded,
+ return String.format("%,d %s of memory requested; %s set is %,d %s. If the file is not corrupt, consider increasing the memory limit.", memoryNeeded,
scale, boundary, memoryLimit, scale);
}
- private static long check(final long request, final long max, final long memory, String memoryType, final String scale) throws MemoryLimitException {
+ private static long check(final long request, final long max, final long memory, final String memoryType, final String scale) throws MemoryLimitException {
check(request, max, scale, MAXIMUM);
check(request, memory, scale, memoryType);
return request;
diff --git a/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java b/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
index 0d7db4738..1a8afc1fa 100644
--- a/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
+++ b/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
@@ -66,6 +66,22 @@ public static int addExact(final int x, final long y) throws ArchiveException {
}
}
+ /**
+ * Delegates to {@link Math#toIntExact(long)} wrapping its {@link ArithmeticException} in our {@link ArchiveException}.
+ *
+ * @param value the long value.
+ * @return the argument as an int.
+ * @throws ArchiveException if the {@code argument} overflows an int.
+ * @since 1.29.0
+ */
+ public static int toIntExact(final long value) throws ArchiveException {
+ try {
+ return Math.toIntExact(value);
+ } catch (final ArithmeticException e) {
+ throw new ArchiveException(e);
+ }
+ }
+
/**
* Delegates to {@link Math#addExact(long, long)} wrapping its {@link ArithmeticException} in our {@link ArchiveException}.
*
diff --git a/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java b/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
index f1e3c90fe..5ca354bbf 100644
--- a/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
+++ b/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
@@ -404,7 +404,7 @@ private CpioArchiveEntry readNewEntry(final boolean hasCrc) throws IOException {
throw new ArchiveException("Found illegal entry with negative name length");
}
newEntry.setChksum(readAsciiLong(8, 16));
- final String name = readCString((int) namesize);
+ final String name = readCString(ArchiveException.toIntExact(namesize));
newEntry.setName(name);
if (CpioUtil.fileType(mode) == 0 && !name.equals(CPIO_TRAILER)) {
throw new ArchiveException(
diff --git a/src/main/java/org/apache/commons/compress/utils/IOUtils.java b/src/main/java/org/apache/commons/compress/utils/IOUtils.java
index d6e96fa04..c5747b15c 100644
--- a/src/main/java/org/apache/commons/compress/utils/IOUtils.java
+++ b/src/main/java/org/apache/commons/compress/utils/IOUtils.java
@@ -30,6 +30,7 @@
import java.nio.file.Files;
import java.nio.file.LinkOption;
+import org.apache.commons.compress.MemoryLimitException;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.output.NullOutputStream;
@@ -47,6 +48,11 @@ public final class IOUtils {
*/
public static final LinkOption[] EMPTY_LINK_OPTIONS = {};
+ /**
+ * The {@code SOFT_MAX_ARRAY_LENGTH} constant from Java's internal ArraySupport class.
+ */
+ private static final int SOFT_MAX_ARRAY_LENGTH = Integer.MAX_VALUE - 8;
+
/**
* Closes the given Closeable and swallows any IOException that may occur.
*
@@ -109,15 +115,15 @@ public static long copy(final InputStream input, final OutputStream output, fina
*
* @param input the InputStream to copy
* @param output the target Stream
- * @param len maximum amount of bytes to copy
+ * @param length maximum amount of bytes to copy
* @return the number of bytes copied
* @throws IOException if an error occurs
* @since 1.21
* @deprecated Use {@link org.apache.commons.io.IOUtils#copyLarge(InputStream, OutputStream, long, long)}.
*/
@Deprecated
- public static long copyRange(final InputStream input, final long len, final OutputStream output) throws IOException {
- return org.apache.commons.io.IOUtils.copyLarge(input, output, 0, len);
+ public static long copyRange(final InputStream input, final long length, final OutputStream output) throws IOException {
+ return org.apache.commons.io.IOUtils.copyLarge(input, output, 0, length);
}
/**
@@ -229,7 +235,7 @@ public static void readFully(final ReadableByteChannel channel, final ByteBuffer
*/
public static byte[] readRange(final InputStream input, final int length) throws IOException {
final ByteArrayOutputStream output = new ByteArrayOutputStream();
- org.apache.commons.io.IOUtils.copyLarge(input, output, 0, length);
+ org.apache.commons.io.IOUtils.copyLarge(input, output, 0, MemoryLimitException.checkBytes(length, SOFT_MAX_ARRAY_LENGTH));
return output.toByteArray();
}
diff --git a/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java b/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
index 7520ed6c0..2a0490909 100644
--- a/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
+++ b/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
@@ -23,12 +23,13 @@
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.io.ByteArrayInputStream;
-import java.io.EOFException;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import org.apache.commons.compress.AbstractTest;
+import org.apache.commons.compress.MemoryLimitException;
+import org.apache.commons.compress.archivers.ArchiveException;
import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.Test;
@@ -105,7 +106,7 @@ void testEndOfFileInEntry_c_namesize_0x7FFFFFFF() throws Exception {
final byte[] data = new byte[header.getBytes(StandardCharsets.US_ASCII).length + 1];
System.arraycopy(header.getBytes(), 0, data, 0, header.getBytes().length);
try (CpioArchiveInputStream cpio = new CpioArchiveInputStream(new ByteArrayInputStream(data))) {
- assertThrows(EOFException.class, () -> cpio.getNextEntry());
+ assertThrows(MemoryLimitException.class, () -> cpio.getNextEntry());
}
}
@@ -132,7 +133,7 @@ void testEndOfFileInEntry_c_namesize_0xFFFFFFFF() throws Exception {
final byte[] data = new byte[header.getBytes(StandardCharsets.US_ASCII).length + 1];
System.arraycopy(header.getBytes(), 0, data, 0, header.getBytes().length);
try (CpioArchiveInputStream cpio = new CpioArchiveInputStream(new ByteArrayInputStream(data))) {
- assertThrows(EOFException.class, () -> cpio.getNextEntry());
+ assertThrows(ArchiveException.class, () -> cpio.getNextEntry());
}
}