You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by bu...@apache.org on 2008/09/11 14:14:47 UTC
DO NOT REPLY [Bug 45778] New: Cloning autofilters leads to HUGE files
https://issues.apache.org/bugzilla/show_bug.cgi?id=45778
Summary: Cloning autofilters leads to HUGE files
Product: POI
Version: unspecified
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: normal
Priority: P2
Component: HSSF
AssignedTo: dev@poi.apache.org
ReportedBy: antti.koskimaki@joinex.com
Created an attachment (id=22554)
--> (https://issues.apache.org/bugzilla/attachment.cgi?id=22554)
excel and sourcecode to reproduce situation
Cloning sheets having several columns with autofilter leads to HUGE output
files.
Opening the file into excel and saving again reduces size dramatically.
Cloning same sheets without autofilters produces much smaller file.
- without autofilter 320k
- with autofilter 13080k
- same back from excel 580k
Size-effect is the same with both svn r696683 (working autofilters, see fix
45720) as with 3.0.2-FINAL (broken autofilters).
Sourcecode to create examples attached.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
DO NOT REPLY [Bug 45778] Cloning autofilters leads to HUGE files
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=45778
Josh Micich <jo...@gildedtree.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |FIXED
--- Comment #3 from Josh Micich <jo...@gildedtree.com> 2009-01-28 18:40:31 PST ---
(In reply to comment #2)
BTW - the buffer overrun problem in RecordInputStream (that helped keep this
bug silent) was fixed in svn r707778 ( https://svn.apache.org/viewcvs.cgi?view=rev&rev=707778 ).
It would be better if we close off this bugzilla to help track things more
clearly. I am pretty sure that the original bug is fixed. The new bug you
describe has different characteristics and the fix will probably be unrelated
to svn r707450 ( https://svn.apache.org/viewcvs.cgi?view=rev&rev=707450 ).
Please open a new bugzilla for your second bug. I suspect that this new bug
will be of a lower priority since most POI users won't clone anywhere near 83
sheets. However, if you need a fix urgently, you can speed things up by
providing a patch or at least identifying the exact cause of the bug.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
DO NOT REPLY [Bug 45778] Cloning autofilters leads to HUGE files
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=45778
Antti Koskimäki <an...@joinex.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |
--- Comment #2 from Antti Koskimäki <an...@joinex.com> 2008-10-24 00:05:28 PST ---
> I re-ran your test (cloning 50 sheets from a 22KB input workbook).
> Here are the resulting file sizes:
> before fix: 13,101KB
> after fix: 588KB
Thanks, verified that this is OK.
But, when I tried to stress-test the fix, I noticed that by adding more
cloning-iterations the workbook got corrupted at some point (limit=83, to be
exact :=). Workbook without auto-filters seems not to corrupt, not even with
1000 clones.
When I tried to open the workbook I got several "File error, data may have been
lost" pop-ups in row, one for each (!) cloned sheet. Besides fatal, also very
annoying :=)
Not sure how closely relates to this bug or fix, seems to re-produce with
3.2-FINAL too. But you mentioning buffer-overruns and me using the same
test-case to reproduce, plus auto-filters having something to do with it, I
decided to re-open this bug instead of reporting new one.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
DO NOT REPLY [Bug 45778] Cloning autofilters leads to HUGE files
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=45778
Josh Micich <jo...@gildedtree.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
--- Comment #1 from Josh Micich <jo...@gildedtree.com> 2008-10-23 12:29:45 PST ---
fixed in svn r707450
Junit added.
ObjRecord was not reading sub-record ftLbsData properly. ftLbsData does not
encode its length in the second ushort field. This field often has a value
like 0x1FFEE. Due to another bug in RecordInputStream that allows buffer
read-overruns, the ftLbsData was being interpreted as being nearly 8KB instead
of 20 bytes.
I re-ran your test (cloning 50 sheets from a 22KB input workbook).
Here are the resulting file sizes:
before fix: 13,101KB
after fix: 588KB
Your test spreadsheet had 30 (autofilter) combo-boxes, which get cloned 50
times. Assuming 8KB extra per combo-box, this would account for roughly 12MB
extra allocation. I didn't attempt the same test (manually) with Excel, but
the remaining difference from the 580KB that you got could be explained by
POI's expansion of MULRK records. Preserving MULRK, MULBLANK and SHARREDFMLA
records is on the to-do list but currently not a high priority.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org