You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@poi.apache.org by lazikid <ra...@hotmail.com> on 2009/11/10 02:33:41 UTC
POI-XSSF Memory Issue
I modified the sample given in the Upgrading to POI 3.5 (usermodel converting
code) as below to add some instrumentation lines :
=========================
// Define a few rows
System.out.println(" max memory to be used " +
Runtime.getRuntime().maxMemory()/(1024 * 1024) + "M");
for(int rownum = 0; rownum < 70000; rownum++) {
if (rownum % 1000 ==0) {
System.out.println("row " + rownum );
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
long consumedMemory = (totalMemory - freeMemory);
System.out.println(" total memory " + totalMemory/(1024 * 1024) +
"M");
System.out.println(" free memory " + freeMemory/(1024 * 1024) + "M");
System.out.println(" consumedMemory " + consumedMemory/(1024 * 1024)
+ "M");
System.out.println(" +++++++++++++++++++++++++++ ");
}
......
.......
}
=========================
I ran it with java -Xmx1924M and this is what I got :
max memory to be used 1710M
row 0
total memory 11M
free memory 6M
consumedMemory 5M
+++++++++++++++++++++++++++
row 1000
total memory 80M
free memory 38M
consumedMemory 41M
+++++++++++++++++++++++++++
row 2000
total memory 120M
free memory 29M
consumedMemory 91M
+++++++++++++++++++++++++++
row 3000
total memory 169M
free memory 53M
consumedMemory 115M
+++++++++++++++++++++++++++
......
......
row 37000
total memory 1635M
free memory 260M
consumedMemory 1374M
+++++++++++++++++++++++++++
row 38000
total memory 1635M
free memory 218M
consumedMemory 1416M
+++++++++++++++++++++++++++
row 39000
total memory 1635M
free memory 237M
consumedMemory 1397M
+++++++++++++++++++++++++++
row 40000
total memory 1635M
free memory 214M
consumedMemory 1421M
+++++++++++++++++++++++++++
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.ArrayList.<init>(Unknown Source)
at java.util.ArrayList.<init>(Unknown Source)
at org.apache.xmlbeans.impl.store.Saver.<init>(Saver.java:86)
at
org.apache.xmlbeans.impl.store.Saver$SynthNamespaceSaver.<init>(Saver.java:867)
at org.apache.xmlbeans.impl.store.Saver.<init>(Saver.java:121)
at
org.apache.xmlbeans.impl.store.Saver$TextSaver.<init>(Saver.java:896)
at org.apache.xmlbeans.impl.store.Cursor._xmlText(Cursor.java:546)
at org.apache.xmlbeans.impl.store.Cursor.xmlText(Cursor.java:2436)
at
org.apache.xmlbeans.impl.values.XmlObjectBase.xmlText(XmlObjectBase.java:1455)
at
org.apache.xmlbeans.impl.values.XmlObjectBase.toString(XmlObjectBase.java:1440)
at
org.apache.poi.xssf.model.SharedStringsTable.addEntry(SharedStringsTable.java:161)
at
org.apache.poi.xssf.usermodel.XSSFCell.setCellValue(XSSFCell.java:314)
at com.ramzeen.poi.first.FirstPOI.main(FirstPOI.java:85)
This is too much memory consumption, isn't it ?
--
View this message in context: http://old.nabble.com/POI-XSSF-Memory-Issue-tp26276869p26276869.html
Sent from the POI - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: POI-XSSF Memory Issue
Posted by David Fisher <df...@jmlafferty.com>.
Your results are as expected. POI will consume memory in the usermodel
because it currently needs to load the whole file into memory.
If you are only looking to pull the data from the file then you should
look at using the eventmodel.
Take a look at http://poi.apache.org/spreadsheet/how-to.html
Regards,
Dave
On Nov 9, 2009, at 5:33 PM, lazikid wrote:
>
> I modified the sample given in the Upgrading to POI 3.5 (usermodel
> converting
> code) as below to add some instrumentation lines :
>
> =========================
> // Define a few rows
> System.out.println(" max memory to be used " +
> Runtime.getRuntime().maxMemory()/(1024 * 1024) + "M");
> for(int rownum = 0; rownum < 70000; rownum++) {
> if (rownum % 1000 ==0) {
> System.out.println("row " + rownum );
> long freeMemory = Runtime.getRuntime().freeMemory();
> long totalMemory = Runtime.getRuntime().totalMemory();
> long consumedMemory = (totalMemory - freeMemory);
> System.out.println(" total memory " + totalMemory/(1024 *
> 1024) +
> "M");
> System.out.println(" free memory " + freeMemory/(1024 * 1024)
> + "M");
> System.out.println(" consumedMemory " + consumedMemory/(1024
> * 1024)
> + "M");
>
> System.out.println(" +++++++++++++++++++++++++++ ");
> }
> ......
> .......
>
> }
> =========================
>
> I ran it with java -Xmx1924M and this is what I got :
>
> max memory to be used 1710M
> row 0
> total memory 11M
> free memory 6M
> consumedMemory 5M
> +++++++++++++++++++++++++++
> row 1000
> total memory 80M
> free memory 38M
> consumedMemory 41M
> +++++++++++++++++++++++++++
> row 2000
> total memory 120M
> free memory 29M
> consumedMemory 91M
> +++++++++++++++++++++++++++
> row 3000
> total memory 169M
> free memory 53M
> consumedMemory 115M
> +++++++++++++++++++++++++++
>
> ......
> ......
>
> row 37000
> total memory 1635M
> free memory 260M
> consumedMemory 1374M
> +++++++++++++++++++++++++++
> row 38000
> total memory 1635M
> free memory 218M
> consumedMemory 1416M
> +++++++++++++++++++++++++++
> row 39000
> total memory 1635M
> free memory 237M
> consumedMemory 1397M
> +++++++++++++++++++++++++++
> row 40000
> total memory 1635M
> free memory 214M
> consumedMemory 1421M
> +++++++++++++++++++++++++++
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.<init>(Unknown Source)
> at java.util.ArrayList.<init>(Unknown Source)
> at org.apache.xmlbeans.impl.store.Saver.<init>(Saver.java:86)
> at
> org.apache.xmlbeans.impl.store.Saver
> $SynthNamespaceSaver.<init>(Saver.java:867)
> at org.apache.xmlbeans.impl.store.Saver.<init>(Saver.java:121)
> at
> org.apache.xmlbeans.impl.store.Saver$TextSaver.<init>(Saver.java:896)
> at org.apache.xmlbeans.impl.store.Cursor._xmlText(Cursor.java:
> 546)
> at org.apache.xmlbeans.impl.store.Cursor.xmlText(Cursor.java:
> 2436)
> at
> org
> .apache
> .xmlbeans.impl.values.XmlObjectBase.xmlText(XmlObjectBase.java:1455)
> at
> org
> .apache
> .xmlbeans.impl.values.XmlObjectBase.toString(XmlObjectBase.java:1440)
> at
> org
> .apache
> .poi.xssf.model.SharedStringsTable.addEntry(SharedStringsTable.java:
> 161)
> at
> org.apache.poi.xssf.usermodel.XSSFCell.setCellValue(XSSFCell.java:314)
> at com.ramzeen.poi.first.FirstPOI.main(FirstPOI.java:85)
>
> This is too much memory consumption, isn't it ?
>
> --
> View this message in context: http://old.nabble.com/POI-XSSF-Memory-Issue-tp26276869p26276869.html
> Sent from the POI - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org