You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2014/09/09 17:39:12 UTC
[Bug 56936] New: [PATCH] Upload of PPTX causes very high memory
usage leading to system instability
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
Bug ID: 56936
Summary: [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Product: POI
Version: 3.9-FINAL
Hardware: All
OS: All
Status: NEW
Severity: critical
Priority: P2
Component: HSLF
Assignee: dev@poi.apache.org
Reporter: Dmitry.Velichkevich@synesis.ru
Created attachment 31978
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31978&action=edit
The patch for the POI OOXML library (tar.gz archive, created via the 'ant -f
patch.xml' command)
Hi.
We experience high memory consumption during analysis certain PPTX documents by
POI 3.9-beta1. Unfortunately, it is not easy to create such a problematic
document. And we cannot attach our problematic documents, because they contain
valuable information. Also we don’t know the version of MS Office where these
documents were created.
Analysis has shown that the problem is mainly in the slides with the formula
element. Elements of formula contain symbols in ‘UTF-16LE’ encoding such as ‘
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #2 from Dmitry <Dm...@synesis.ru> ---
Created attachment 31980
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31980&action=edit
Screenshot of profile metrics of problematic PPTX analyzing before patching
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #8 from Andreas Beeker <an...@gmx.de> ---
Created attachment 32141
--> https://issues.apache.org/bugzilla/attachment.cgi?id=32141&action=edit
custom xml options patch
How about not hard coding the xmlreader inside the xslfslide instead providing
a mechanism to generally override the used xmloption for all classes?
(see TestXSLFBugs.bug56936())
Please give it a try and see if it works for you.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
Andreas Beeker <an...@gmx.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #4 from Dmitry <Dm...@synesis.ru> ---
(In reply to Dmitry from comment #0)
> Created attachment 31978 [details]
> The patch for the POI OOXML library (tar.gz archive, created via the 'ant -f
> patch.xml' command)
>
> Hi.
>
> We experience high memory consumption during analysis certain PPTX documents
> by POI 3.9-beta1. Unfortunately, it is not easy to create such a problematic
> document. And we cannot attach our problematic documents, because they
> contain valuable information. Also we don’t know the version of MS Office
> where these documents were created.
>
> Analysis has shown that the problem is mainly in the slides with the formula
> element. Elements of formula contain symbols in ‘UTF-16LE’ encoding such as ‘
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #1 from Dmitry <Dm...@synesis.ru> ---
Created attachment 31979
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31979&action=edit
Thread stack trace dumps of problematic PPTX analyzing (plain text file)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #3 from Dmitry <Dm...@synesis.ru> ---
Created attachment 31981
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31981&action=edit
Screenshot of profile metrics of problematic PPTX analyzing after patching
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 56936] [PATCH] Upload of PPTX causes very high memory usage
leading to system instability
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56936
--- Comment #7 from Dmitry <Dm...@synesis.ru> ---
I apologize for the incomplete description and empty comments. I tried to send
sample characters in the ‘UTF-16LE’ encoding, but all the text beginning with
those characters was cut.
Hi.
We experience high memory consumption during analysis certain PPTX documents by
POI 3.9-beta1. Unfortunately, it is not easy to create such a problematic
document. And we cannot attach our problematic documents, because they contain
valuable information. Also we don’t know the version of MS Office where these
documents were created.
Analysis has shown that the problem is mainly in the slides with the formula
element. Elements of formula contain symbols in ‘UTF-16LE’ encoding. The
presentation may have several slides with the formula, but only some specific
slide causes high memory consumption.
The cause of this problem is SAX parser called Piccolo which is used by
‘XmlBeans’ by default. 'CharUtil' at Piccolo incorrectly processes text of the
problematic slides and allocating more and more memory until OOM is reached. We
did not perform in-depth investigation of this problem. Thread stack trace
dumps are attached to a ticket as 'pptx-analysis-stack-trace.txt' (plain text
file).
We tried to figure out how to set up another XML reader for POI, but we
stumbled upon the following limitation in the POI source code
(org.apache.poi.xslf.usermodel.XSLFSlide):
/**
* Construct a SpreadsheetML slide from a package part
*
* @param part the package part holding the slide data,
* the content type must be
<code>application/vnd.openxmlformats-officedocument.slide+xml</code>
* @param rel the package relationship holding this slide,
* the relationship type must be
http://schemas.openxmlformats.org/officeDocument/2006/relationships/slide
*/
XSLFSlide(PackagePart part, PackageRelationship rel) throws IOException,
XmlException {
super(part, rel);
SldDocument doc =
SldDocument.Factory.parse(getPackagePart().getInputStream());
_slide = doc.getSld();
setCommonSlideData(_slide.getCSld());
}
Execution is delegated to the ‘XmlBeans’ implementation in this constructor.
And this implementation uses ‘XmlOptions’ to get configuration of XML API
(‘SAXParserFactory’ and ‘XMLReader’ implementions and so on). But ‘XSLFSlide’
does not put ‘XmlOptions’ parameter to ‘SldDocument.Factory.parse()’ method.
Hence, ‘XmlBeans’ always selects Piccolo parser.
We have performed some tests and have come up with a patch to ‘poi-ooxml’ lib
(specifically the ‘XSLFSlide’ class), where the Xerces SAX parser is used
instead of the Piccolo SAX parser in the class. This change dramatically
effects the system resources and no spike is seen in memory at all (compare
‘pptx-analysis-before-patching.jpg’ with ‘pptx-analysis-after-patching.jpg’
profiling metrics screenshots attached to the ticket).
We have attached the patch as TAR.GZ archive called ‘patch.tar.gz’ (created
using ‘ant -f patch.xml’ command). We hope very much that this patch will be
applied in future versions of POI, because POI 3.10-FINAL also does not allow
set up XML API.
But the best option is the ability to customize XML API implementation for POI
in accordance with the following documentation (including SAXParserFactory,
XMLReader and so on):
http://docs.oracle.com/javase/7/docs/api/javax/xml/parsers/SAXParserFactory.html#newInstance%28%29
Please, let me know if I should report this kind of improvement as a separate
ticket.
Looking forward for your input.
Regards,
Dmitry
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org