You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Gary D. Gregory (Jira)" <ji...@apache.org> on 2023/01/23 11:15:00 UTC
[jira] [Updated] (COMPRESS-639) Crash when adding multiple files with the same path and when Zip64Mode is in use.
[ https://issues.apache.org/jira/browse/COMPRESS-639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary D. Gregory updated COMPRESS-639:
-------------------------------------
Affects Version/s: (was: 1.23)
> Crash when adding multiple files with the same path and when Zip64Mode is in use.
> ---------------------------------------------------------------------------------
>
> Key: COMPRESS-639
> URL: https://issues.apache.org/jira/browse/COMPRESS-639
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers, Compressors
> Affects Versions: 1.21, 1.22
> Environment: Tested on MacBook Pro 2019 (2.6 GHz 6-Core Intel Core i7, 32GB DDR4)
> MacOS 13.1
> JDK 11.0.13
> Tested with commons-compress 1.21, 1.22 and 1.23-SNAPSHOT
> Reporter: Andrew Gawron
> Priority: Major
>
> Crash when adding 2 zip entries to a large archive. The entries had the same name.
> After the investigation we found out that ZipArchiveOutputStream has a race condition. When adding two entries with the same entry name an entry is being added to _entries_ LinkedList and then again it is being added to _metaData_ HashMap. If the modification time ({_}race condition{_} here), name and other params are the same then the metaData is not being updated for the second entry. Then when createCentralFileHeader iterates over _entries_ the first entry is being found in metaData keyset. It gets modified later by adding extras. Then second entry tries to find its metadata but it fails because metaData key has been changed.
> Potential solution: container keys should be immutable and they should not be modified after being added to the container.
> Sample code that triggers exception:
> {code:java}
> @Test
> public void shouldThrowDueToRaceConditionInZipArchiveOutputStream() throws IOException, ExecutionException, InterruptedException {
> var testOutputStream = new ByteArrayOutputStream();
> String fileContent = "A";
> final int NUM_OF_FILES = 100;
> var inputStreams = new LinkedList<InputStream>();
> for (int i = 0; i < NUM_OF_FILES; i++) {
> inputStreams.add(new ByteArrayInputStream(fileContent.getBytes(StandardCharsets.UTF_8)));
> }
> var zipCreator = new ParallelScatterZipCreator();
> var zipArchiveOutputStream = new ZipArchiveOutputStream(testOutputStream);
> zipArchiveOutputStream.setUseZip64(Zip64Mode.Always);
> for (int i = 0; i < inputStreams.size(); i++) {
> ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry("./dir/myfile.txt");
> zipArchiveEntry.setMethod(ZipEntry.DEFLATED);
> final var inputStream = inputStreams.get(i);
> zipCreator.addArchiveEntry(zipArchiveEntry, () -> inputStream);
> }
> zipCreator.writeTo(zipArchiveOutputStream);
> zipArchiveOutputStream.close(); // it will throw NullPointerException here
> } {code}
> Exception:
> {code:java}
> /* java.lang.NullPointerException at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream$EntryMetaData.access$800(ZipArchiveOutputStream.java:1998) at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.createCentralFileHeader(ZipArchiveOutputStream.java:1356) at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeCentralDirectoryInChunks(ZipArchiveOutputStream.java:580) at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.finish(ZipArchiveOutputStream.java:546) at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.close(ZipArchiveOutputStream.java:1090) at com.xxx.yyy.impl.backuprestore.backup.container.StreamZipWriterTest.shouldThrowDueToRaceConditionInZipArchiveOutputStream(StreamZipWriterTest.java:130) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.mockito.internal.runners.DefaultInternalRunner$1$1.evaluate(DefaultInternalRunner.java:55) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.mockito.internal.runners.DefaultInternalRunner$1.run(DefaultInternalRunner.java:100) at org.mockito.internal.runners.DefaultInternalRunner.run(DefaultInternalRunner.java:107) at org.mockito.internal.runners.StrictRunner.run(StrictRunner.java:41) at org.mockito.junit.MockitoJUnitRunner.run(MockitoJUnitRunner.java:163) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54) */ {code}
> Workaround:
> Add a unique comment for each file so it will make the entry always unique (ZipArchiveEntry#addComment)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)