[Bug 57572] New: Extensive comments easily trigger OOM

[bu...@apache.org]

https://issues.apache.org/bugzilla/show_bug.cgi?id=57572

            Bug ID: 57572
           Summary: Extensive comments easily trigger OOM
           Product: POI
           Version: 3.11-FINAL
          Hardware: PC
            Status: NEW
          Severity: critical
          Priority: P2
         Component: SXSSF
          Assignee: dev@poi.apache.org
          Reporter: genka_v@yahoo.com

Created attachment 32458
  --> https://issues.apache.org/bugzilla/attachment.cgi?id=32458&action=edit
Test harness based on your original impl.

Multiple comments easily lead to OOM.

See attached test to reproduce. 
It should be run as follows 
... -Xmx512M -Dexcel.comments=all SSPerformanceTest SXSSF 10000 6 1

Above, we generate a fairly short comment for every single cell, 60,000
comments total, which isn't a big deal, from the business app perspective.

The OOM stack always points to allocation of XML objects related to comments
and anchors, such as:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit
exceeded
    at org.apache.xmlbeans.impl.store.Xobj.ensureParent(Xobj.java:614)
    at org.apache.xmlbeans.impl.store.Xobj.getNormal(Xobj.java:661)
    at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1958)
    at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1846)
    at org.apache.xmlbeans.impl.store.Cursor.insertNode(Cursor.java:189)
    at
org.apache.xmlbeans.impl.store.Cursor._insertAttributeWithValue(Cursor.java:1765)
    at
org.apache.xmlbeans.impl.store.Cursor._insertAttributeWithValue(Cursor.java:1755)
    at
org.apache.xmlbeans.impl.store.Cursor.insertAttributeWithValue(Cursor.java:4271)
    at
org.apache.poi.xssf.usermodel.XSSFVMLDrawing.write(XSSFVMLDrawing.java:152)
    at
org.apache.poi.xssf.usermodel.XSSFVMLDrawing.commit(XSSFVMLDrawing.java:176)
    at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:323)
    at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:327)
    at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:327)
    at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:195)
    at
org.apache.poi.xssf.streaming.SXSSFWorkbook.write(SXSSFWorkbook.java:856)
    at SSPerformanceTest.main(SSPerformanceTest.java:132)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org

[Bug 57572] Extensive comments easily trigger OOM

[bu...@apache.org]

https://issues.apache.org/bugzilla/show_bug.cgi?id=57572

--- Comment #1 from Nick Burch <ap...@gagravarr.org> ---
As a general guide, SXSSF relies on the common (in-memory) XSSF code for the
bulk of the file format handling. The only exceptions to that are the sheet
contents (buffered), and shared strings table (optional on-disk storage, see
patches, not yet committed). 

Normally, this trade-off works - lots of things can be easily supported without
massive re-writes, by using the existing XSSF code, and the other parts of the
.xlsx file format are normally much much smaller

It looks like that in your case, one of these assumptions breaks down...

If having that many comments is a requirements, sadly it's going to need some
more work, most likely from yourselves as you're the only ones affected. At a
quick glance, it would seem your options are:
 * Provide an alternate XSSFVMLDrawing (and friends?) implementation which
works in a streaming manner, much as SXSSFSheet does
 * Provide an alternate XSSFVMLDrawing (and friends?) implementation which uses
an on-disk storage for random access, and a low-memory write, much as the
proposed shared strings table support does

Jump on the dev list if you need help with starting down either route!

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org

[Bug 57572] Extensive comments easily trigger OOM

[bu...@apache.org]

https://issues.apache.org/bugzilla/show_bug.cgi?id=57572

Gennadiy Vaysman <ge...@yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 OS|                            |All

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org