You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2020/12/30 07:13:21 UTC
[Bug 65042] New: Writing pircture to workbook cause memory leak due
to picture data is to much in memory.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
Bug ID: 65042
Summary: Writing pircture to workbook cause memory leak due to
picture data is to much in memory.
Product: POI
Version: 4.1.2-FINAL
Hardware: Macintosh
OS: Mac OS X 10.4
Status: NEW
Severity: blocker
Priority: P2
Component: SXSSF
Assignee: dev@poi.apache.org
Reporter: satioasuka@gmail.com
Target Milestone: ---
Created attachment 37670
--> https://bz.apache.org/bugzilla/attachment.cgi?id=37670&action=edit
Memory Usage Dump
Sxssf using Xssf handle its picture. So picture will save in memory, not flush
with row. This means if we have a lot of pictures had to write in xlsx, this
will cause memory leak which was I am struggling with. I can't find a way to
flush picture data before workbook.write().
The test case is, 300kb picture, one row write 10 pictures. JVM -Xmx128m.
There will be an oom when write 50 rows. In that time, the picture data list
will have 60mb in memory.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
Dominik Stadler <do...@gmx.at> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Dominik Stadler <do...@gmx.at> ---
Can you provide a small piece of code which reproduces the problem for you?
Ideally as a self-contained unit-test so we can reproduce the problem and take
a closer look?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
--- Comment #7 from PJ Fanning <fa...@yahoo.com> ---
I added r1894203 - still experimental/beta - needs testing still - may be
removed or modified
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
--- Comment #3 from Runzhi <sa...@gmail.com> ---
Sorry for the late response.
I have created a repository that can reproduce the problem.
Run unit test with max heap size 128m.
Repository URL is :
https://github.com/foresx/poi-memory-leak-demo
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
Dominik Stadler <do...@gmx.at> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
Severity|major |enhancement
--- Comment #4 from Dominik Stadler <do...@gmx.at> ---
Thanks for the detailed reproducing code, I took a look at your sample-project
now.
Pictures in .xlsx files are not stored per "row" or "sheet", but rather
globally in a separate structure along the other parts.
The current SXSSFWorkbook only flushes and removes rows based on the
"rowAccesswindowSize".
So flushing picture data for SXSSFWorkbook is currently not supported, we can
consider adding it as an enhancement, naturally it will happen sooner if you
can propose an implementation that offers this as additional option for
SXSSFWorkbook in some way, however it will require some coding as you likely
need to flush out pictures in a similar way as the rows and then a write-time
combine the information into the final document as well.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
--- Comment #5 from Runzhi <sa...@gmail.com> ---
Thanks a lot. When I have free time, I think I will have a try.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
Dominik Stadler <do...@gmx.at> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Writing pircture to |Adding pictures to workbook
|workbook cause memory leak |causes memory leak
|due to picture data is to |
|much in memory. |
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Writing pircture to workbook cause memory leak due to
picture data is to much in memory.
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
--- Comment #1 from Runzhi <sa...@gmail.com> ---
Created attachment 37671
--> https://bz.apache.org/bugzilla/attachment.cgi?id=37671&action=edit
Memory I count in program
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
PJ Fanning <fa...@yahoo.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|NEW |RESOLVED
--- Comment #8 from PJ Fanning <fa...@yahoo.com> ---
The features to change ZipPackage to use temp files to save memory will be a
beta feature in POI 5.1.0
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Writing pircture to workbook cause memory leak due to
picture data is to much in memory.
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
Dominik Stadler <do...@gmx.at> changed:
What |Removed |Added
----------------------------------------------------------------------------
Severity|blocker |major
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
[Bug 65042] Adding pictures to workbook causes memory leak
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65042
--- Comment #6 from PJ Fanning <fa...@yahoo.com> ---
I think the issue is that we don't have a TempFilePackagePart that can
optionally be used instead of MemoryPackagePart.
https://poi.apache.org/apidocs/dev/org/apache/poi/openxml4j/opc/ZipPackage.html#createPartImpl-org.apache.poi.openxml4j.opc.PackagePartName-java.lang.String-boolean-
This would allow us to avoid using memory (while slowing things down by using
temp files).
The other issue is how to configure the code so that it can choose whether to
use MemoryPackagePart or TempFilePackagePart.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org