You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2017/01/31 08:03:06 UTC
[Bug 60670] New: Fails to create document nodes in MSI files larger
than 2 GB
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
Bug ID: 60670
Summary: Fails to create document nodes in MSI files larger
than 2 GB
Product: POI
Version: 3.15-FINAL
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: POIFS
Assignee: dev@poi.apache.org
Reporter: marcus.lundblad@primekey.se
Target Milestone: ---
I have generated 2 MSI files 3 GB in size (one with a single large document
entry and one with three smaller 1 GB entries).
Trying to add an additional document entry in the file system, I get:
Caused by: java.lang.IllegalArgumentException: Negative position
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:755)
[rt.jar:1.8.0_121]
at
org.apache.poi.poifs.nio.FileBackedDataSource.write(FileBackedDataSource.java:120)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.createBlockIfNeeded(NPOIFSFileSystem.java:505)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422)
[poi-3.15.jar:3.15]
at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.createDocument(NPOIFSFileSystem.java:689)
[poi-3.15.jar:3.15]
at
org.signserver.module.msauthcode.signer.MSAuthCodeSigner.signMSI(MSAuthCodeSigner.java:570)
[SignServer-Module-MSAuthCode-4.1.0-SNAPSHOT.jar:]
I think the culprit here is the prevBlock and nextBlock index "pointers" in
StreamBlockByteBuffer being ints causing an overflow.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
Nick Burch <ap...@gagravarr.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #1 from Nick Burch <ap...@gagravarr.org> ---
I can't seem to find the source code for MSAuthCodeSigner, so it's hard to tell
if this is a POI bug or a mistake in the calling code. (It seems to be missing
from the module list in
https://svn.cesecore.eu/svn/signserver/trunk/signserver/modules/ )
Are you able to share the POI-related lines of code from there, so we can take
a look? (If it is a bug, we'll need to write a unit test, which'll require the
code steps taken to reproduce)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #8 from Marcus Lundblad <ma...@primekey.se> ---
(In reply to Nick Burch from comment #7)
> I've just tried with your program and your 3gb msi file, but I think the
> file might be corrupt? When I run the program, it fails with:
>
> Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 8,
> Size: 8
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at
> org.apache.poi.poifs.property.PropertyTableBase.
> populatePropertyTree(PropertyTableBase.java:128)
> at
> org.apache.poi.poifs.property.PropertyTableBase.<init>(PropertyTableBase.
> java:63)
> at
> org.apache.poi.poifs.property.NPropertyTable.<init>(NPropertyTable.java:66)
> at
> org.apache.poi.poifs.filesystem.NPOIFSFileSystem.
> readCoreContents(NPOIFSFileSystem.java:440)
> at
> org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.
> java:235)
> at
> org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.
> java:168)
> at Main.main(Main.java:18)
>
> I've done a fix for PropertyTableBase to skip broken entries. That then lets
> me run your program without error
I can still reproduce the "Negative position" exception problem with a file
generated by the WiX toolset.
I only managed to generate a very large file with random data (so it won't
compress, will see if I can still provide a link to download it).
Also, it did actually work with my sample test program, but when I increased
the size of the byte array to add as an extra document to 10000 bytes I got the
same error again, so I guess it might be due to padding (when a new block needs
to be created).
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
Markus Kilås <jm...@kilas.se> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jmarkus+apache01@kilas.se
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #2 from Marcus Lundblad <ma...@primekey.se> ---
Right, missed that, sorry about that.
This is in a module in-development.
The code looks something like this:
final NPOIFSFileSystem fsOut =
new NPOIFSFileSystem(responseData.getAsFile(), false)) {
// Add the signature file
fsOut.createDocument(new ByteArrayInputStream(signedbytes),
"\05DigitalSignature");
where the getAsFile() call would return a File instance pointing to the
resulting file (which is just a copy of the original file).
and the signedbytes is a byte array with the signature data (I don't think the
content should really matter in this case.
I could try to write a simple stand-alone program to expose the issue as well.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #4 from Nick Burch <ap...@gagravarr.org> ---
If you could knock up a failing unit test based on a temp File and some huge
empty starting entries, that'd be great!
If not, I can try later
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #7 from Nick Burch <ap...@gagravarr.org> ---
I've just tried with your program and your 3gb msi file, but I think the file
might be corrupt? When I run the program, it fails with:
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 8, Size:
8
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.poi.poifs.property.PropertyTableBase.populatePropertyTree(PropertyTableBase.java:128)
at
org.apache.poi.poifs.property.PropertyTableBase.<init>(PropertyTableBase.java:63)
at
org.apache.poi.poifs.property.NPropertyTable.<init>(NPropertyTable.java:66)
at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.readCoreContents(NPOIFSFileSystem.java:440)
at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:235)
at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:168)
at Main.main(Main.java:18)
I've done a fix for PropertyTableBase to skip broken entries. That then lets me
run your program without error
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #5 from Marcus Lundblad <ma...@primekey.se> ---
Created attachment 34697
--> https://bz.apache.org/bugzilla/attachment.cgi?id=34697&action=edit
Stand-alone program to expose bug
The supplied program can be run with an input file as commandline argument.
The file is overwritten (if successful).
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #3 from Andreas Beeker <ki...@apache.org> ---
Btw. I've received a contribution from Sebastian S. a while ago, which I
eventually plan to integrate ... maybe it's a help to you ...
http://people.apache.org/~kiwiwings/20140908_MSIAPI_JUnitTests.zip
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 60670] Fails to create document nodes in MSI files larger than
2 GB
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60670
--- Comment #6 from Marcus Lundblad <ma...@primekey.se> ---
(In reply to Marcus Lundblad from comment #5)
> Created attachment 34697 [details]
> Stand-alone program to expose bug
>
> The supplied program can be run with an input file as commandline argument.
> The file is overwritten (if successful).
https://www.update.uu.se/~ml/large.zip
An archive containing a very large MSI file.
NOTE: the file will decompress to a very large file.
The MSI file was created with msibuilder (from msitools) and only contains
document streams filled with zeros. It will not actually work as a real
installer in Windows.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org