You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@poi.apache.org by Hongdi Ren <ry...@gmail.com> on 2016/09/05 02:44:52 UTC
Re: 2G unreachable objects when using xssf
Hi all,
We’re using poi & poi-ooxml 3.14 generating tens of thousands of rows to be downloaded.
We have a problem that jvm keeps old gc 5 times/min but nothing could be collected. What’s more, when we looked into the jvmdump, it has 2G unreachable objects closed related with poi. We have no idea why that can’t be cleaned out.
It might be better to use the stream api which we would try recently, but is there any idea why could this happen?
Attached detailed information:
code
And the jvmdump shows
Re: 2G unreachable objects when using xssf
Posted by Hongdi Ren <ry...@gmail.com>.
Thanks very much for the suggestions. I’ll try them
On 9/5/16, 1:08 PM, "Javen O'Neal" <ja...@gmail.com> wrote:
By the way, if your production code really is as simple as the example you
provided, the only change you need to make to get this to work for
streaming is
Workbook wb = new SXSSFWorkbook(1);
Keep in mind that SXSSFWorkbooks don't use a shared string table--instead
they embed the string literal as the cell value. As such, RichText strings
are not supported in SXSSF.
On Sep 4, 2016 22:01, "Javen O'Neal" <ja...@gmail.com> wrote:
> Did you try os.close() and wb.close() after your flush?
>
> On Sep 4, 2016 20:50, "Hongdi Ren" <ry...@gmail.com> wrote:
>
>> Hi Javen,
>>
>> Per my understanding, before call Workbook.write, the XML nodes shouldn’t
>> be unreachable, the workbook should have strong reference to the nodes. Is
>> that right?
>>
>> Seems the attached picture isn’t working. Here’s our code. The output
>> stream is from servlet response. The valueList(rows) is about tens of
>> thousands. The nameList(column) is about 20.
>>
>> public static void export(OutputStream os, List<String> nameList,
>> List<List<String>> valueList) throws IOException{
>> Workbook wb = new XSSFWorkbook();
>> String safeSheetNameName = WorkbookUtil.createSafeSheetNa
>> me("sheet1");
>> Sheet sheet = wb.createSheet(safeSheetNameName);
>> Row nameRow = sheet.createRow(0);
>> for (int i = 0; i < nameList.size(); i++) {
>> nameRow.createCell(i).setCellValue(nameList.get(i));
>> }
>> for (int i = 1; i <= valueList.size(); i++) {
>> List<String> list = valueList.get(i - 1);
>> Row valueRow = sheet.createRow(i);
>> for (int j = 0; j < list.size(); j++) {
>> valueRow.createCell(j).setCellValue(list.get(j));
>> }
>> }
>> wb.write(os);
>> os.flush();
>> }
>>
>> On 9/5/16, 11:25 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
>>
>> Without more detail, my guess would be that your JVM has 2GB of XML
>> nodes.
>> Nothing is flushed to disk with XSSF until you call Workbook.write,
>> and
>> nothing is cleared until Workbook.close.
>>
>> If you only need to create a workbook, the SXSSF streaming API is the
>> way
>> to go.
>>
>> On Sep 4, 2016 7:45 PM, "Hongdi Ren" <ry...@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> >
>> >
>> > We’re using poi & poi-ooxml 3.14 generating tens of thousands of
>> rows to
>> > be downloaded.
>> >
>> >
>> >
>> > We have a problem that jvm keeps old gc 5 times/min but nothing
>> could be
>> > collected. What’s more, when we looked into the jvmdump, it has 2G
>> > unreachable objects closed related with poi. We have no idea why
>> that can’t
>> > be cleaned out.
>> >
>> >
>> >
>> > It might be better to use the stream api which we would try
>> recently, but
>> > is there any idea why could this happen?
>> >
>> >
>> >
>> >
>> >
>> > Attached detailed information:
>> >
>> > code
>> >
>> >
>> >
>> > And the jvmdump shows
>> >
>> >
>> >
>> > ------------------------------------------------------------
>> ---------
>> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> > For additional commands, e-mail: user-help@poi.apache.org
>> >
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: 2G unreachable objects when using xssf
Posted by Javen O'Neal <ja...@gmail.com>.
By the way, if your production code really is as simple as the example you
provided, the only change you need to make to get this to work for
streaming is
Workbook wb = new SXSSFWorkbook(1);
Keep in mind that SXSSFWorkbooks don't use a shared string table--instead
they embed the string literal as the cell value. As such, RichText strings
are not supported in SXSSF.
On Sep 4, 2016 22:01, "Javen O'Neal" <ja...@gmail.com> wrote:
> Did you try os.close() and wb.close() after your flush?
>
> On Sep 4, 2016 20:50, "Hongdi Ren" <ry...@gmail.com> wrote:
>
>> Hi Javen,
>>
>> Per my understanding, before call Workbook.write, the XML nodes shouldn’t
>> be unreachable, the workbook should have strong reference to the nodes. Is
>> that right?
>>
>> Seems the attached picture isn’t working. Here’s our code. The output
>> stream is from servlet response. The valueList(rows) is about tens of
>> thousands. The nameList(column) is about 20.
>>
>> public static void export(OutputStream os, List<String> nameList,
>> List<List<String>> valueList) throws IOException{
>> Workbook wb = new XSSFWorkbook();
>> String safeSheetNameName = WorkbookUtil.createSafeSheetNa
>> me("sheet1");
>> Sheet sheet = wb.createSheet(safeSheetNameName);
>> Row nameRow = sheet.createRow(0);
>> for (int i = 0; i < nameList.size(); i++) {
>> nameRow.createCell(i).setCellValue(nameList.get(i));
>> }
>> for (int i = 1; i <= valueList.size(); i++) {
>> List<String> list = valueList.get(i - 1);
>> Row valueRow = sheet.createRow(i);
>> for (int j = 0; j < list.size(); j++) {
>> valueRow.createCell(j).setCellValue(list.get(j));
>> }
>> }
>> wb.write(os);
>> os.flush();
>> }
>>
>> On 9/5/16, 11:25 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
>>
>> Without more detail, my guess would be that your JVM has 2GB of XML
>> nodes.
>> Nothing is flushed to disk with XSSF until you call Workbook.write,
>> and
>> nothing is cleared until Workbook.close.
>>
>> If you only need to create a workbook, the SXSSF streaming API is the
>> way
>> to go.
>>
>> On Sep 4, 2016 7:45 PM, "Hongdi Ren" <ry...@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> >
>> >
>> > We’re using poi & poi-ooxml 3.14 generating tens of thousands of
>> rows to
>> > be downloaded.
>> >
>> >
>> >
>> > We have a problem that jvm keeps old gc 5 times/min but nothing
>> could be
>> > collected. What’s more, when we looked into the jvmdump, it has 2G
>> > unreachable objects closed related with poi. We have no idea why
>> that can’t
>> > be cleaned out.
>> >
>> >
>> >
>> > It might be better to use the stream api which we would try
>> recently, but
>> > is there any idea why could this happen?
>> >
>> >
>> >
>> >
>> >
>> > Attached detailed information:
>> >
>> > code
>> >
>> >
>> >
>> > And the jvmdump shows
>> >
>> >
>> >
>> > ------------------------------------------------------------
>> ---------
>> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> > For additional commands, e-mail: user-help@poi.apache.org
>> >
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
Re: 2G unreachable objects when using xssf
Posted by Javen O'Neal <ja...@gmail.com>.
Did you try os.close() and wb.close() after your flush?
On Sep 4, 2016 20:50, "Hongdi Ren" <ry...@gmail.com> wrote:
> Hi Javen,
>
> Per my understanding, before call Workbook.write, the XML nodes shouldn’t
> be unreachable, the workbook should have strong reference to the nodes. Is
> that right?
>
> Seems the attached picture isn’t working. Here’s our code. The output
> stream is from servlet response. The valueList(rows) is about tens of
> thousands. The nameList(column) is about 20.
>
> public static void export(OutputStream os, List<String> nameList,
> List<List<String>> valueList) throws IOException{
> Workbook wb = new XSSFWorkbook();
> String safeSheetNameName = WorkbookUtil.
> createSafeSheetName("sheet1");
> Sheet sheet = wb.createSheet(safeSheetNameName);
> Row nameRow = sheet.createRow(0);
> for (int i = 0; i < nameList.size(); i++) {
> nameRow.createCell(i).setCellValue(nameList.get(i));
> }
> for (int i = 1; i <= valueList.size(); i++) {
> List<String> list = valueList.get(i - 1);
> Row valueRow = sheet.createRow(i);
> for (int j = 0; j < list.size(); j++) {
> valueRow.createCell(j).setCellValue(list.get(j));
> }
> }
> wb.write(os);
> os.flush();
> }
>
> On 9/5/16, 11:25 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
>
> Without more detail, my guess would be that your JVM has 2GB of XML
> nodes.
> Nothing is flushed to disk with XSSF until you call Workbook.write, and
> nothing is cleared until Workbook.close.
>
> If you only need to create a workbook, the SXSSF streaming API is the
> way
> to go.
>
> On Sep 4, 2016 7:45 PM, "Hongdi Ren" <ry...@gmail.com> wrote:
>
> > Hi all,
> >
> >
> >
> > We’re using poi & poi-ooxml 3.14 generating tens of thousands of
> rows to
> > be downloaded.
> >
> >
> >
> > We have a problem that jvm keeps old gc 5 times/min but nothing
> could be
> > collected. What’s more, when we looked into the jvmdump, it has 2G
> > unreachable objects closed related with poi. We have no idea why
> that can’t
> > be cleaned out.
> >
> >
> >
> > It might be better to use the stream api which we would try
> recently, but
> > is there any idea why could this happen?
> >
> >
> >
> >
> >
> > Attached detailed information:
> >
> > code
> >
> >
> >
> > And the jvmdump shows
> >
> >
> >
> > ------------------------------------------------------------
> ---------
> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> > For additional commands, e-mail: user-help@poi.apache.org
> >
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
>
Re: 2G unreachable objects when using xssf
Posted by Hongdi Ren <ry...@gmail.com>.
Hi Javen,
Per my understanding, before call Workbook.write, the XML nodes shouldn’t be unreachable, the workbook should have strong reference to the nodes. Is that right?
Seems the attached picture isn’t working. Here’s our code. The output stream is from servlet response. The valueList(rows) is about tens of thousands. The nameList(column) is about 20.
public static void export(OutputStream os, List<String> nameList, List<List<String>> valueList) throws IOException{
Workbook wb = new XSSFWorkbook();
String safeSheetNameName = WorkbookUtil.createSafeSheetName("sheet1");
Sheet sheet = wb.createSheet(safeSheetNameName);
Row nameRow = sheet.createRow(0);
for (int i = 0; i < nameList.size(); i++) {
nameRow.createCell(i).setCellValue(nameList.get(i));
}
for (int i = 1; i <= valueList.size(); i++) {
List<String> list = valueList.get(i - 1);
Row valueRow = sheet.createRow(i);
for (int j = 0; j < list.size(); j++) {
valueRow.createCell(j).setCellValue(list.get(j));
}
}
wb.write(os);
os.flush();
}
On 9/5/16, 11:25 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
Without more detail, my guess would be that your JVM has 2GB of XML nodes.
Nothing is flushed to disk with XSSF until you call Workbook.write, and
nothing is cleared until Workbook.close.
If you only need to create a workbook, the SXSSF streaming API is the way
to go.
On Sep 4, 2016 7:45 PM, "Hongdi Ren" <ry...@gmail.com> wrote:
> Hi all,
>
>
>
> We’re using poi & poi-ooxml 3.14 generating tens of thousands of rows to
> be downloaded.
>
>
>
> We have a problem that jvm keeps old gc 5 times/min but nothing could be
> collected. What’s more, when we looked into the jvmdump, it has 2G
> unreachable objects closed related with poi. We have no idea why that can’t
> be cleaned out.
>
>
>
> It might be better to use the stream api which we would try recently, but
> is there any idea why could this happen?
>
>
>
>
>
> Attached detailed information:
>
> code
>
>
>
> And the jvmdump shows
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: 2G unreachable objects when using xssf
Posted by Javen O'Neal <ja...@gmail.com>.
Without more detail, my guess would be that your JVM has 2GB of XML nodes.
Nothing is flushed to disk with XSSF until you call Workbook.write, and
nothing is cleared until Workbook.close.
If you only need to create a workbook, the SXSSF streaming API is the way
to go.
On Sep 4, 2016 7:45 PM, "Hongdi Ren" <ry...@gmail.com> wrote:
> Hi all,
>
>
>
> We’re using poi & poi-ooxml 3.14 generating tens of thousands of rows to
> be downloaded.
>
>
>
> We have a problem that jvm keeps old gc 5 times/min but nothing could be
> collected. What’s more, when we looked into the jvmdump, it has 2G
> unreachable objects closed related with poi. We have no idea why that can’t
> be cleaned out.
>
>
>
> It might be better to use the stream api which we would try recently, but
> is there any idea why could this happen?
>
>
>
>
>
> Attached detailed information:
>
> code
>
>
>
> And the jvmdump shows
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>