You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2016/02/24 03:41:18 UTC
[Bug 59058] New: OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
Bug ID: 59058
Summary: OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
Product: POI
Version: unspecified
Hardware: PC
Status: NEW
Severity: normal
Priority: P2
Component: OPC
Assignee: dev@poi.apache.org
Reporter: tallison@mitre.org
Shawn Johnson recently posted a smallish docx file on TIKA-1866 that causes an
OOM.
WARNING: trying to parse this file in Intellij caused a system crash and
required a hard reboot on Windows.
I can reproduce this in pure POI with the following:
OPCPackage pkg = OPCPackage.open(path)
System.out.println("before creating extractor");
POIXMLTextExtractor poiExtractor =
ExtractorFactory.createExtractor(pkg);
System.out.println("finished creating extractor");
The OOM happens during createExtractor, and I never hit the second println.
However, there is no OOM with:
OPCPackage pkg = OPCPackage.open(Files.newInputStream(path));
System.out.println("before creating extractor");
POIXMLTextExtractor poiExtractor =
ExtractorFactory.createExtractor(pkg);
System.out.println("finished creating extractor");
Any idea what might cause the different treatment?
Java 1.8.0_72 on Windows.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when parsing docx after OPCPackage.open with File
but not with InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #6 from Tim Allison <ta...@mitre.org> ---
I reused Dominik's test on 57031.
If we use xerces instead of piccolo, we don't appear to have a problem with
parsing either 57031 or 59058.
This leads to an OOM for both files:
ZipFile zf = new ZipFile(path0.toAbsolutePath().toString());
ZipEntry entry = zf.getEntry("word/document.xml");
DocumentDocument document =
DocumentDocument.Factory.parse(zf.getInputStream(entry));
assertNotNull(document);
This works for both files:
ZipFile zf = new ZipFile(path1.toAbsolutePath().toString());
ZipEntry entry = zf.getEntry("word/document.xml");
XMLInputFactory xmlif = XMLInputFactory.newInstance();
XMLStreamReader reader =
xmlif.createXMLStreamReader(zf.getInputStream(entry));
DocumentDocument document = DocumentDocument.Factory.parse(reader);
assertNotNull(document);
zf.close();
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #1 from Tim Allison <ta...@mitre.org> ---
Created attachment 33585
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33585&action=edit
Triggering file submitted by Shawn Johnson on TIKA-1866
When I re-save this file, it no longer triggers the problem.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
Nick Burch <ap...@gagravarr.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
OS| |All
--- Comment #2 from Nick Burch <ap...@gagravarr.org> ---
Other than all the contents of the zip having the date "1980-01-01 00:00", I
can't see anything immediately wrong
Loading the sampe file as an OPCPackage from an InputStream on my machine seems
to use ~13mb of memory (from a couple of tests). Loading it from a File is
around ~3mb.
So, nothing obvious springs to mind. Would someone be able to dig in and find
out where the memory is going, and how it differs between the two cases?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when parsing docx after OPCPackage.open with File
but not with InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
Tim Allison <ta...@mitre.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |DUPLICATE
Status|NEEDINFO |RESOLVED
--- Comment #8 from Tim Allison <ta...@mitre.org> ---
*** This bug has been marked as a duplicate of bug 57031 ***
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #4 from Nick Burch <ap...@gagravarr.org> ---
I'm not seeing the OOM on opening the package, only on extraction. Opening the
package isn't taking much memory. JVM is java version "1.7.0_95" on Ubuntu
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when parsing docx after OPCPackage.open with File
but not with InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #5 from Tim Allison <ta...@mitre.org> ---
Of course, sorry. Thank you.
This looks very similar to 57031. However, with the 57031.docx file, I'm
getting an OOM while parsing whether I open the package with a File or with an
InputStream.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when parsing docx after OPCPackage.open with File
but not with InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #7 from Tim Allison <ta...@mitre.org> ---
In XWPFDocument's onDocumentRead(), if we change:
DocumentDocument doc =
DocumentDocument.Factory.parse(getPackagePart().getInputStream(),
DEFAULT_XML_OPTIONS);
to:
XMLInputFactory xmlif = XMLInputFactory.newInstance();
XMLStreamReader reader =
xmlif.createXMLStreamReader(getPackagePart().getInputStream());
DocumentDocument doc = DocumentDocument.Factory.parse(reader,
DEFAULT_XML_OPTIONS);
We can parse both files.
This change is on the periphery of my competence. Any problems with this? How
can we require xerces via .newInstance()/newFactory()?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when parsing docx after OPCPackage.open with File
but not with InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
Tim Allison <ta...@mitre.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|OOM when opening OPCPackage |OOM when parsing docx after
|with File but not with |OPCPackage.open with File
|InputStream (TIKA-1866) |but not with InputStream
| |(TIKA-1866)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 59058] OOM when opening OPCPackage with File but not with
InputStream (TIKA-1866)
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59058
--- Comment #3 from Tim Allison <ta...@mitre.org> ---
Interesting. Thank you for taking a look! You're not seeing the OOM, then...
What version of Java? Windows...right?
In addition to Windows with:
java version "1.8.0_72"
Java(TM) SE Runtime Environment (build 1.8.0_72-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode)
I'm also getting the OOM with Tika (at least) in RHEL with:
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org