You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by gg...@apache.org on 2025/08/06 12:37:49 UTC

(commons-compress) branch master updated: org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory

This is an automated email from the ASF dual-hosted git repository.

ggregory pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-compress.git


The following commit(s) were added to refs/heads/master by this push:
     new d59a49ff8 org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory
d59a49ff8 is described below

commit d59a49ff8b47f25462caa8f20d18472120b4e9c7
Author: Gary D. Gregory <ga...@gmail.com>
AuthorDate: Wed Aug 6 08:37:43 2025 -0400

    org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry()
    now throws a MemoryLimitException instead of OutOfMemoryError when it
    can't process input greater than available memory
    
    - Add org.apache.commons.compress.archivers.ArchiveException.toIntExact(long)
---
 src/changes/changes.xml                                  |  6 ++++--
 .../apache/commons/compress/MemoryLimitException.java    | 13 +++++++++++--
 .../commons/compress/archivers/ArchiveException.java     | 16 ++++++++++++++++
 .../compress/archivers/cpio/CpioArchiveInputStream.java  |  2 +-
 .../java/org/apache/commons/compress/utils/IOUtils.java  | 14 ++++++++++----
 .../archivers/cpio/CpioArchiveInputStreamTest.java       |  7 ++++---
 6 files changed, 46 insertions(+), 12 deletions(-)

diff --git a/src/changes/changes.xml b/src/changes/changes.xml
index abf242be9..433f3b5b0 100644
--- a/src/changes/changes.xml
+++ b/src/changes/changes.xml
@@ -54,7 +54,7 @@ The <action> type attribute can be add,update,fix,remove.
       <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">Fix format string in org.apache.commons.compress.archivers.sevenz.SevenZFile.assertFitsIntoNonNegativeInt(String, long).</action>
       <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.sevenz.SevenZFile.Builder.get() now throws a MemoryLimitException (an IOException subclass) instead of an OOME.</action>
       <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream constructors now throws CompressorException instead of ArrayIndexOutOfBoundsException.</action>
-      <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.TapeInputStream.resetBlockSize(int, boolean) now throws a MemoryLimitException (an IOException subclass) instead of an OOME.</action>
+      <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.TapeInputStream.resetBlockSize(int, boolean) now throws a MemoryLimitException (an IOException subclass) instead of an OutOfMemoryError.</action>
       <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.dump.DumpArchiveInputStream.getNextEntry() throws an ArchiveException instead of ArrayIndexOutOfBoundsException.</action>
       <action type="fix" dev="ggregory" due-to="Dominik Stadler, Gary Gregory" issue="COMPRESS-598">org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(byte[], int, int) now throws an IOException instead of a NullPointerException.</action>
       <action type="fix" dev="ggregory" due-to="Tyler Nighswander, Gary Gregory">org.apache.commons.compress.archivers.zip.ZipFile.createBoundedInputStream(long, long) now throws an ArchiveException instead of IllegalArgumentException.</action>
@@ -72,7 +72,8 @@ The <action> type attribute can be add,update,fix,remove.
       <action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.archivers.zip.ExplodingInputStream now throws ArchiveException instead of Arithmetic exception.</action>
       <action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.harmony.pack200.RunCodec now throws ArchiveException instead of Arithmetic exception.</action>
       <action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.harmony.pack200.FileBands now throws ArchiveException instead of Arithmetic exception.</action>
-      <action type="fix" dev="ggregory" due-to="Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveEntry now throws ArchiveException instead of Arithmetic exception.</action>
+      <action type="fix" dev="ggregory" due-to="Stan, Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveEntry now throws ArchiveException instead of Arithmetic exception.</action>
+      <action type="fix" dev="ggregory" due-to="Stan, Gary Gregory">org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextEntry() now throws a MemoryLimitException instead of OutOfMemoryError when it can't process input greater than available memory.</action>
       <!-- ADD -->
       <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.MemoryLimitException.MemoryLimitException(long, long).</action>
       <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.CompressException.CompressException(String, Object...).</action>
@@ -84,6 +85,7 @@ The <action> type attribute can be add,update,fix,remove.
       <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(int, int).</action>
       <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(int, long).</action>
       <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.addExact(long, long).</action>
+      <action type="add" dev="ggregory" due-to="Gary Gregory">Add org.apache.commons.compress.archivers.ArchiveException.toIntExact(long).</action>
       <!-- UPDATE -->
     </release>
     <release version="1.28.0" date="2025-07-26" description="This is a feature and maintenance release. Java 8 or later is required.">
diff --git a/src/main/java/org/apache/commons/compress/MemoryLimitException.java b/src/main/java/org/apache/commons/compress/MemoryLimitException.java
index 1fbf7f3c1..31614100b 100644
--- a/src/main/java/org/apache/commons/compress/MemoryLimitException.java
+++ b/src/main/java/org/apache/commons/compress/MemoryLimitException.java
@@ -19,6 +19,8 @@
 
 package org.apache.commons.compress;
 
+import java.io.ByteArrayOutputStream;
+
 /**
  * If a stream checks for estimated memory allocation, and the estimate goes above the memory limit, this is thrown. This can also be thrown if a stream tries
  * to allocate a byte array that is larger than the allowable limit.
@@ -27,17 +29,24 @@
  */
 public class MemoryLimitException extends CompressException {
 
+    /**
+     * The maximum array size defined privately in {@link ByteArrayOutputStream}.
+     *
+     * @since 1.29.0
+     */
+    public static int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
+
     private static final String MAXIMUM = "maximum";
     private static final String BYTES = "bytes";
     private static final String KIB = "KiB";
     private static final long serialVersionUID = 1L;
 
     private static String buildMessage(final long memoryNeeded, final long memoryLimit, final String scale, final String boundary) {
-        return String.format("%,d %s of memory would be needed; %s was %,d %s. If the file is not corrupt, consider increasing the memory limit.", memoryNeeded,
+        return String.format("%,d %s of memory requested; %s set is %,d %s. If the file is not corrupt, consider increasing the memory limit.", memoryNeeded,
                 scale, boundary, memoryLimit, scale);
     }
 
-    private static long check(final long request, final long max, final long memory, String memoryType, final String scale) throws MemoryLimitException {
+    private static long check(final long request, final long max, final long memory, final String memoryType, final String scale) throws MemoryLimitException {
         check(request, max, scale, MAXIMUM);
         check(request, memory, scale, memoryType);
         return request;
diff --git a/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java b/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
index 0d7db4738..1a8afc1fa 100644
--- a/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
+++ b/src/main/java/org/apache/commons/compress/archivers/ArchiveException.java
@@ -66,6 +66,22 @@ public static int addExact(final int x, final long y) throws ArchiveException {
         }
     }
 
+    /**
+     * Delegates to {@link Math#toIntExact(long)} wrapping its {@link ArithmeticException} in our {@link ArchiveException}.
+     *
+     * @param value the long value.
+     * @return the argument as an int.
+     * @throws ArchiveException if the {@code argument} overflows an int.
+     * @since 1.29.0
+     */
+    public static int toIntExact(final long value) throws ArchiveException {
+        try {
+            return Math.toIntExact(value);
+        } catch (final ArithmeticException e) {
+            throw new ArchiveException(e);
+        }
+    }
+
     /**
      * Delegates to {@link Math#addExact(long, long)} wrapping its {@link ArithmeticException} in our {@link ArchiveException}.
      *
diff --git a/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java b/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
index f1e3c90fe..5ca354bbf 100644
--- a/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
+++ b/src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStream.java
@@ -404,7 +404,7 @@ private CpioArchiveEntry readNewEntry(final boolean hasCrc) throws IOException {
             throw new ArchiveException("Found illegal entry with negative name length");
         }
         newEntry.setChksum(readAsciiLong(8, 16));
-        final String name = readCString((int) namesize);
+        final String name = readCString(ArchiveException.toIntExact(namesize));
         newEntry.setName(name);
         if (CpioUtil.fileType(mode) == 0 && !name.equals(CPIO_TRAILER)) {
             throw new ArchiveException(
diff --git a/src/main/java/org/apache/commons/compress/utils/IOUtils.java b/src/main/java/org/apache/commons/compress/utils/IOUtils.java
index d6e96fa04..c5747b15c 100644
--- a/src/main/java/org/apache/commons/compress/utils/IOUtils.java
+++ b/src/main/java/org/apache/commons/compress/utils/IOUtils.java
@@ -30,6 +30,7 @@
 import java.nio.file.Files;
 import java.nio.file.LinkOption;
 
+import org.apache.commons.compress.MemoryLimitException;
 import org.apache.commons.io.FileUtils;
 import org.apache.commons.io.output.NullOutputStream;
 
@@ -47,6 +48,11 @@ public final class IOUtils {
      */
     public static final LinkOption[] EMPTY_LINK_OPTIONS = {};
 
+    /**
+     * The {@code SOFT_MAX_ARRAY_LENGTH} constant from Java's internal ArraySupport class.
+     */
+    private static final int SOFT_MAX_ARRAY_LENGTH = Integer.MAX_VALUE - 8;
+
     /**
      * Closes the given Closeable and swallows any IOException that may occur.
      *
@@ -109,15 +115,15 @@ public static long copy(final InputStream input, final OutputStream output, fina
      *
      * @param input  the InputStream to copy
      * @param output the target Stream
-     * @param len    maximum amount of bytes to copy
+     * @param length    maximum amount of bytes to copy
      * @return the number of bytes copied
      * @throws IOException if an error occurs
      * @since 1.21
      * @deprecated Use {@link org.apache.commons.io.IOUtils#copyLarge(InputStream, OutputStream, long, long)}.
      */
     @Deprecated
-    public static long copyRange(final InputStream input, final long len, final OutputStream output) throws IOException {
-        return org.apache.commons.io.IOUtils.copyLarge(input, output, 0, len);
+    public static long copyRange(final InputStream input, final long length, final OutputStream output) throws IOException {
+        return org.apache.commons.io.IOUtils.copyLarge(input, output, 0, length);
     }
 
     /**
@@ -229,7 +235,7 @@ public static void readFully(final ReadableByteChannel channel, final ByteBuffer
      */
     public static byte[] readRange(final InputStream input, final int length) throws IOException {
         final ByteArrayOutputStream output = new ByteArrayOutputStream();
-        org.apache.commons.io.IOUtils.copyLarge(input, output, 0, length);
+        org.apache.commons.io.IOUtils.copyLarge(input, output, 0, MemoryLimitException.checkBytes(length, SOFT_MAX_ARRAY_LENGTH));
         return output.toByteArray();
     }
 
diff --git a/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java b/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
index 7520ed6c0..2a0490909 100644
--- a/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
+++ b/src/test/java/org/apache/commons/compress/archivers/cpio/CpioArchiveInputStreamTest.java
@@ -23,12 +23,13 @@
 import static org.junit.jupiter.api.Assertions.assertThrows;
 
 import java.io.ByteArrayInputStream;
-import java.io.EOFException;
 import java.io.IOException;
 import java.io.InputStream;
 import java.nio.charset.StandardCharsets;
 
 import org.apache.commons.compress.AbstractTest;
+import org.apache.commons.compress.MemoryLimitException;
+import org.apache.commons.compress.archivers.ArchiveException;
 import org.apache.commons.io.IOUtils;
 import org.junit.jupiter.api.Test;
 
@@ -105,7 +106,7 @@ void testEndOfFileInEntry_c_namesize_0x7FFFFFFF() throws Exception {
         final byte[] data = new byte[header.getBytes(StandardCharsets.US_ASCII).length + 1];
         System.arraycopy(header.getBytes(), 0, data, 0, header.getBytes().length);
         try (CpioArchiveInputStream cpio = new CpioArchiveInputStream(new ByteArrayInputStream(data))) {
-            assertThrows(EOFException.class, () -> cpio.getNextEntry());
+            assertThrows(MemoryLimitException.class, () -> cpio.getNextEntry());
         }
     }
 
@@ -132,7 +133,7 @@ void testEndOfFileInEntry_c_namesize_0xFFFFFFFF() throws Exception {
         final byte[] data = new byte[header.getBytes(StandardCharsets.US_ASCII).length + 1];
         System.arraycopy(header.getBytes(), 0, data, 0, header.getBytes().length);
         try (CpioArchiveInputStream cpio = new CpioArchiveInputStream(new ByteArrayInputStream(data))) {
-            assertThrows(EOFException.class, () -> cpio.getNextEntry());
+            assertThrows(ArchiveException.class, () -> cpio.getNextEntry());
         }
     }