RE: [Bug 57042] Writing Large Excel Sheet (~150K rows) causing the api to dump large xml file at /tmp dir.

["Banerjee, Tapas" <Ta...@fmr.com>]

Hi Nick, Apache POI Library writers,

Please refer to our bug report below. Here is the recap of our tries and some of our questions, appreciate if anyone can clarify.

The issue was, when we tried large query (~140K Excel rows) using streaming SXSSF api with POI 3.10Final jar, it dumped huge (few GB) XML file at our server /tmp/ dir causing the disk full.  Smaller query   < 100K works fine.

- As you suggested, we tried POI 11- Beta 2, but we faced the problem XML jar incompatibility issue.  We were using xml_api.jar and Xerce 2.6.1.  After some APi developer suggested,  we upgraded   to Xerces-J-bin.2.11.0.  jar - that problem was resolved.

- We also coded to change the location of tmp dir to a different file system that got more space

- However, when we deployed our application at server - we faced another problem(see below), perhaps related to incompatible jar issue.

We plan to deploy the new SXSSF API at the production trading environment , we want to ensure the application & component we are using are stable.

Questions we have:
_________________
1) From poi developer forum positing I see multiple bugs are reported in  POI 11- Beta 2, most of them fixed at POI 11- Beta 3. Also there is upcoming POI 11 - Final release. 
Which poi (e.g. 3.11 Final) jar version you recommend us to test and use?

2) I see from poi developer forum discussions, some XML component is being separated from the poi jar. What is the name and correct version for the component?

3) What are the dependencies (name & version) for other jars if we use  POI 11.x ?

4) When is most stable version of POI 11 supposed to come out?

Feel free to reach us if you have any questions, in regards to our applications.

Thanks,
-Tapas
Tapas.banerjee@fmr.com





 


-----Original Message-----
From: Nick Burch [mailto:apache@gagravarr.org] 
Sent: Friday, October 17, 2014 3:29 AM
To: POI Developers List
Subject: RE: [Bug 57042] Writing Large Excel Sheet (~150K rows) causing the api to dump large xml file at /tmp dir. Also does not compress & cleanup up causing the /tmp/ disk space full

On Thu, 16 Oct 2014, Banerjee, Tapas wrote:
> Understood.  I tried with intension some calls may be supported by older 
> version of jar, which are deprecated in newer version. Also after 
> googling, I saw someone else also had the same issue, solved by changing 
> jar file. 
> http://stackoverflow.com/questions/26298654/read-and-write-excel-file-using-apache-poi-in-java

>From the looks of it, they solved it by upgrading to a newer version of 
POI and adding all the jars

> Interesting is - this works in my local Window machine. However, when I 
> deploy to Linux server problem appears.

Sounds like you don't actually have the same jars in use on the two 
machines. Check what else is on your classpath, especially what else 
might've been snuck away on the classpath by something else (eg 
application server)

Nick

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org

RE: [Bug 57042] Writing Large Excel Sheet (~150K rows) causing the api to dump large xml file at /tmp dir.

[kiwiwings <ki...@apache.org>]

Banerjee, Tapas wrote
> 1) From poi developer forum positing I see multiple bugs are reported in 
> POI 11- Beta 2, most of them fixed at POI 11- Beta 3. Also there is
> upcoming POI 11 - Final release. 
> Which poi (e.g. 3.11 Final) jar version you recommend us to test and use?

I'd say: test with the development trunk and wait at least for Beta 3 for
the production use.
We planed to release the final before or while the apache con ... there
aren't many days left ...


Banerjee, Tapas wrote
> 2) I see from poi developer forum discussions, some XML component is being
> separated from the poi jar. What is the name and correct version for the
> component?

see http://poi.apache.org/overview.html#components
Normally you'd need poi, poi-ooxml and poi-ooxml-schemas all of the same
version, e.g. 3.11-beta2
I'm not sure, if you mean the discussion about xml schema classes for
encryption/signature, but
you won't need that classes and otherwise the necessary classes will be in
poi-ooxml-schemas.

If you are curious, the ooxml-security artifact - which you don't need - is
in the repo-to-be-staged at:
http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/poi/ooxml-security/


Banerjee, Tapas wrote
> 3) What are the dependencies (name & version) for other jars if we use 
> POI 11.x ?

You won't necessarily need exactly the same versions, but the trunk works
with these:
https://svn.apache.org/repos/asf/poi/trunk/.classpath


Banerjee, Tapas wrote
> 4) When is most stable version of POI 11 supposed to come out?

soon ... 

Andi



--
View this message in context: http://apache-poi.1045710.n5.nabble.com/RE-Bug-57042-Writing-Large-Excel-Sheet-150K-rows-causing-the-api-to-dump-large-xml-file-at-tmp-dir-tp5716970p5716971.html
Sent from the POI - Dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org