You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@poi.apache.org by jayrp11 <ni...@gmail.com> on 2009/10/21 13:17:26 UTC
Memory and perfomance issues
Hi,
As we know at present in current final release 3.5 of POI we have memory and
performance related issues.
I know these has been discussed several times earlier. But now POI has
released final version and those issue may have different take.
So, for this memory and performance related issues does POI has any plan to
address them in coming release.
Can the demo program developed by Yogor, BigGridDemo.java be considered for
solution to this problems?
We have developed our own solution which is inspired by BigGridDemo and it
is working fine. Also I have seen some other peoples are trying the same
thing. With this approach maintainability will be a concern. Like for
upgrade to newer version of POI.
I feel Excel 2007 support is more or less about providing support for large
data, But unfortunately, with current release developers will face troubles
for supporting large Excel 2007 format.
Please let us know about your comments.
Thanks,
Jaydeep.
--
View this message in context: http://www.nabble.com/Memory-and-perfomance-issues-tp25990601p25990601.html
Sent from the POI - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: Memory and perfomance issues
Posted by jayrp11 <ni...@gmail.com>.
You can look at post at below link
http://www.nabble.com/POI-%3A-Memory-and-Perfomance-ts24756016.html
http://www.nabble.com/POI-%3A-Memory-and-Perfomance-ts24756016.html
AR Williamson wrote:
>
> Can you quantify what you mean by "large data" ?
>
> At what point, does the performance/memory degrade?
>
>
> jayrp11 wrote:
>> I feel Excel 2007 support is more or less about providing support for
>> large
>> data, But unfortunately, with current release developers will face
>> troubles
>> for supporting large Excel 2007 format.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Memory-and-perfomance-issues-tp25990601p25990930.html
Sent from the POI - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: Memory and perfomance issues
Posted by "@lan Williamson" <al...@alanwilliamson.org>.
Can you quantify what you mean by "large data" ?
At what point, does the performance/memory degrade?
jayrp11 wrote:
> I feel Excel 2007 support is more or less about providing support for large
> data, But unfortunately, with current release developers will face troubles
> for supporting large Excel 2007 format.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: Memory and perfomance issues
Posted by Alex Ciarlillo <ci...@ohio.edu>.
I recently worked on a small project where I had to overcome these memory
issues. I also based my final solution off BigGridDemo but with the addition
of being able to work on multiple sheets. Since it was a relatively trivial
and small program the solution was fairly hacked together, but I can foresee
in the future that POI will be extremely valuable for projects in which
memory concerns will also most likely be a concern. My question is, why do
you not consider BigGridDemo a "solution"? Is it simply because it is not
full featured production style code, or is it due to limitations or possible
incompatibilities with how the usermodel currently works? I have only used
POI on one other project and this was before there was even support for
.xslx so my knowledge is fairly limited. But, from my understanding the
memory issue lies within the library used to process xml, should memory
footprint improvements be aimed at this library or would a more reasonable
solution be to replace the library altogether in some way? I know these
questions are fairly broad and probably do not have simple answers, I am
just trying to get an idea of whether or not this is something within my
capability to maybe work on in my spare time. I am encouraged at my job to
contribute back to any open source projects I make use of and finally have
the opportunity and motivation to do so. However, as I said before my
experience with all the features of POI is limited and my knowledge of xml
is fairly cursory so basically I doubt if I could do much better than the
library being used. Sorry if these questions may have been more
appropriately posted to the dev list but it seemed appropriate to the
conversation.
On Wed, Oct 21, 2009 at 8:02 AM, Yegor Kozlov <ye...@dinom.ru> wrote:
> usermodel API is unlikely to change. All memory-related improvements will
> be internal and upgrading to a newer version of POI will not be an issue.
>
> BigGridDemo.java is not a "solution", rather a prototype of a API with
> low-memory footprint. Current version is very simple, it was developed only
> to show the idea.
>
>
>
>> We have developed our own solution which is inspired by BigGridDemo and it
>> is working fine. Also I have seen some other peoples are trying the same
>> thing. With this approach maintainability will be a concern. Like for
>> upgrade to newer version of POI.
>>
>>
> Consider contributing your code to the POI project. It will boost the
> development of this part of XSSF and you will also achieve maximum level of
> compatibility with further versions of POI.
>
> Yegor
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> For additional commands, e-mail: user-help@poi.apache.org
>
>
Re: Memory and perfomance issues
Posted by Aram Mirzadeh <aw...@mbcli.com>.
I've run into the same "error" at least and it turned out to be
surprising enough, illegal characters for XML without being escaped.
Either check the source data coming in or add in all of the checks to
make sure the data you're inserting doesn't have < > & etc...
On 8/20/2010 9:30 AM, dpchandra wrote:
> I used BigGridDemo in my project for generating huge data having nearly 60
> thousand rows. In my project i am trying to generate 3 continuous xlsx files
> continuously. Out of 3 files only two files which has limited rows generated
> without any issue. But the third file which has huge data is giving problem
> while opening. Code wise i checked there are no exceptions and temporary xml
> file was generated. But while opening it is giving an error like
>
> "Replaced Part: /xl/worksheets/sheet1.xml part with XML error. Illegal xml
> character. Line 31287, column 139."
>
> Can any one suggest me what may be the cause for this error.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: Memory and perfomance issues
Posted by dpchandra <dp...@gmail.com>.
I used BigGridDemo in my project for generating huge data having nearly 60
thousand rows. In my project i am trying to generate 3 continuous xlsx files
continuously. Out of 3 files only two files which has limited rows generated
without any issue. But the third file which has huge data is giving problem
while opening. Code wise i checked there are no exceptions and temporary xml
file was generated. But while opening it is giving an error like
"Replaced Part: /xl/worksheets/sheet1.xml part with XML error. Illegal xml
character. Line 31287, column 139."
Can any one suggest me what may be the cause for this error.
--
View this message in context: http://apache-poi.1045710.n5.nabble.com/Memory-and-perfomance-issues-tp2301191p2642176.html
Sent from the POI - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org
Re: Memory and perfomance issues
Posted by Yegor Kozlov <ye...@dinom.ru>.
usermodel API is unlikely to change. All memory-related improvements will be internal and upgrading to a newer version
of POI will not be an issue.
BigGridDemo.java is not a "solution", rather a prototype of a API with low-memory footprint. Current version is very
simple, it was developed only to show the idea.
>
> We have developed our own solution which is inspired by BigGridDemo and it
> is working fine. Also I have seen some other peoples are trying the same
> thing. With this approach maintainability will be a concern. Like for
> upgrade to newer version of POI.
>
Consider contributing your code to the POI project. It will boost the development of this part of XSSF and you will also
achieve maximum level of compatibility with further versions of POI.
Yegor
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org