You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by Kirill Rajbhandary <ki...@jedox.com> on 2019/06/13 14:39:25 UTC
POI memory usage degradation
Dear POI DEV Team,
Unfortunately, I found out that the latest POI library poi-ooxml with versions 4.0.1, 4.1.0 consumes much more memory than the version 3.17 when it is required to write information to already existing large Excel file.
It requires approximately 6GB Java Heap Space to add these rows using poi-ooxml library with version 3.17 but it is not enough even twice more (12GB) with version 4.0.1 or 4.1.0. Besides, the performance is much worse with the latest versions of poi-ooxml. OpenJDK 11.0.02 was used to run this example.
In attachment the corresponding example with ExcelWriter class can be found that tries to add only 5 rows to already existing sheet "Sheet2" having only header row with 16 columns. The first sheet "Sheet1" of this file already contains 240K rows with 23 columns. Zipped ExcelWriter with Test-Template.xslx can be downloaded from the next link:
https://drive.google.com/file/d/1vq6WvMv5mSFUzyJinN9aoYRBStxZuJCq/view
To run ExcelWriter main method, please specify the full path to Test-Template.xlsx and result file path. To change poi-ooxml version, it requires to change it in pom.xml, rebuild the project and run it again.
Could you please advise why it became worse comparing with the old library version poi-ooxml-3.17.jar and how it could be improved?
Best regards,
Kirill
www.jedox.com
Kirill Rajbhandary
Senior Software Engineer
+49 761 15147 237
Jedox AG
Bismarckallee 7a
79098Freiburg im Breisgau
Germany
Executive Board: Florian Winterstein (Chairman), Bernd Eisenblätter, Maximilian Prinz zu Hohenlohe-Waldenburg
Supervisory Board: Bernhard Wöbker (Chairman), Curt Gunsenheimer, Thilo Schmid
Place of Business: Amtsgericht Freiburg HRB 702118
[TM.V06112018]
Re: POI memory usage degradation
Posted by Jörn Franke <jo...@gmail.com>.
Which Java version did you use for 3.17 and 4.1.0.
If you updated Java you may observe more memory consumption.
Additionally, you can save memory and CPU if you read those files using SXSSF mode.
Libraries on top of POI such as Excel Streamer may help you with this:
https://github.com/monitorjbl/excel-streaming-reader
> Am 13.06.2019 um 16:39 schrieb Kirill Rajbhandary <ki...@jedox.com>:
>
> Dear POI DEV Team,
>
> Unfortunately, I found out that the latest POI library poi-ooxml with versions 4.0.1, 4.1.0 consumes much more memory than the version 3.17 when it is required to write information to already existing large Excel file.
> It requires approximately 6GB Java Heap Space to add these rows using poi-ooxml library with version 3.17 but it is not enough even twice more (12GB) with version 4.0.1 or 4.1.0. Besides, the performance is much worse with the latest versions of poi-ooxml. OpenJDK 11.0.02 was used to run this example.
> In attachment the corresponding example with ExcelWriter class can be found that tries to add only 5 rows to already existing sheet "Sheet2" having only header row with 16 columns. The first sheet "Sheet1" of this file already contains 240K rows with 23 columns. Zipped ExcelWriter with Test-Template.xslx can be downloaded from the next link:
>
> https://drive.google.com/file/d/1vq6WvMv5mSFUzyJinN9aoYRBStxZuJCq/view
>
> To run ExcelWriter main method, please specify the full path to Test-Template.xlsx and result file path. To change poi-ooxml version, it requires to change it in pom.xml, rebuild the project and run it again.
>
> Could you please advise why it became worse comparing with the old library version poi-ooxml-3.17.jar and how it could be improved?
>
> Best regards,
> Kirill
>
> www.jedox.com
> Kirill Rajbhandary
> Senior Software Engineer
>
> +49 761 15147 237
> Jedox AG
> Bismarckallee 7a
> 79098
> Freiburg im Breisgau
> Germany
> Executive Board: Florian Winterstein (Chairman), Bernd Eisenblätter, Maximilian Prinz zu Hohenlohe‑Waldenburg
> Supervisory Board: Bernhard Wöbker (Chairman), Curt Gunsenheimer, Thilo Schmid
> Place of Business: Amtsgericht Freiburg HRB 702118
>
> [TM.V06112018]
RE: POI memory usage degradation
Posted by Kirill Rajbhandary <ki...@jedox.com>.
Dear POI DEV Team,
Do you have any updates regarding to the below described issue?
Best regards,
Kirill Rajbhandary
Senior Software Engineer
Jedox AG
www.jedox.com
Kirill Rajbhandary
Senior Software Engineer
+49 761 15147 237
Jedox AG
Bismarckallee 7a
79098Freiburg im Breisgau
Germany
Executive Board: Florian Winterstein (Chairman), Bernd Eisenblätter, Maximilian Prinz zu Hohenlohe-Waldenburg
Supervisory Board: Bernhard Wöbker (Chairman), Curt Gunsenheimer, Thilo Schmid
Place of Business: Amtsgericht Freiburg HRB 702118
[TM.V06112018]
From: Kirill Rajbhandary <ki...@jedox.com>
Sent: Thursday, 13 June, 2019 16:39
To: dev@poi.apache.org
Subject: POI memory usage degradation
Dear POI DEV Team,
Unfortunately, I found out that the latest POI library poi-ooxml with versions 4.0.1, 4.1.0 consumes much more memory than the version 3.17 when it is required to write information to already existing large Excel file.
It requires approximately 6GB Java Heap Space to add these rows using poi-ooxml library with version 3.17 but it is not enough even twice more (12GB) with version 4.0.1 or 4.1.0. Besides, the performance is much worse with the latest versions of poi-ooxml. OpenJDK 11.0.02 was used to run this example.
In attachment the corresponding example with ExcelWriter class can be found that tries to add only 5 rows to already existing sheet "Sheet2" having only header row with 16 columns. The first sheet "Sheet1" of this file already contains 240K rows with 23 columns. Zipped ExcelWriter with Test-Template.xslx can be downloaded from the next link:
https://drive.google.com/file/d/1vq6WvMv5mSFUzyJinN9aoYRBStxZuJCq/view
To run ExcelWriter main method, please specify the full path to Test-Template.xlsx and result file path. To change poi-ooxml version, it requires to change it in pom.xml, rebuild the project and run it again.
Could you please advise why it became worse comparing with the old library version poi-ooxml-3.17.jar and how it could be improved?
Best regards,
Kirill Rajbhandary
Senior Software Engineer
Jedox AG