You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@poi.apache.org by Glen Stampoultzis <gs...@iinet.net.au> on 2004/02/26 03:07:32 UTC
RE: Problem generating a large file when following the
POIFSstandard
You are right: the knowledge is the most valuable part. That's why we
prefer to be paid for that knowledge. Sometimes we give way that knowledge
for free but that's our choice and it's usually to further the cause of the
open source we are involved in and not to act as an unpaid employee.
At 09:49 AM 26/02/2004, you wrote:
>Dear Andrew,
>
>The code itself is only one part of the project. Perhaps even more valuable
>than actual code is the documentation, and knowledge that poeple in this
>group have. Its important to discuss the actual standards themselves. The
>fact that Microsoft dont discuss these things, or make them public is
>EXACTLY the reason why this project is so valuable. Perhaps we shouldn't be
>so quick to turn away people asking questions about how POIFS persistance
>works. Answering these questions will help people understand the code, and
>find bugs, etc.
>
>The service is not free? Perhaps your time is not free, but if people are
>asking questions about how POIFS works, Im happy to provide a free service,
>wherever I can help. The quesiton was not C++ specific, it was merely about
>the POIFS standard, so I think it's perfectly acceptable.
>
>I dont think you should have to donate code in order to get an answer about
>how existing code works, even if you are deriving something from it,
>especially under an Apache license.
>
>-- Kais
>
>-----Original Message-----
>From: Andrew C. Oliver [mailto:acoliver@apache.org]
>Sent: 25 February 2004 22:40
>To: POI Users List
>Subject: Re: Problem generating a large file when following the
>POIFSstandard
>
>
>Unless you plan to donate the C++ code to the project, I'm not sure this is
>the appropriate place to ask your question. The source is open and free,
>the service (like helping you work on a proprietary language fork of POI)
>isn't.
>--
>Andrew C. Oliver
>http://www.superlinksoftware.com/poi.jsp
>Custom enhancements and Commercial Implementation for Jakarta POI
>
>http://jakarta.apache.org/poi
>For Java and Excel, Got POI?
>
>The views expressed in this email are those of the author and are almost
>definitely not shared by the Apache Software Foundation, its board or its
>general membership. In fact they probably most definitively disagree with
>everything espoused in the above email.
>
> > From: "Brandon Belvin" <bb...@iss-md.com>
> > Organization: Information Systems Support, Inc.
> > Reply-To: "POI Users List" <po...@jakarta.apache.org>
> > Date: Wed, 25 Feb 2004 16:04:00 -0600
> > To: "POI User Mailing List" <po...@jakarta.apache.org>
> > Subject: Problem generating a large file when following the POIFS standard
> >
> > As an add-on to an existing database front-end application, we have
>allowed
> > the user to export their report's data to Excel. Following POIFS'
> > standards, we used C++ to write the Excel file. Upon testing, we
>discovered
> > our code works perfectly for files under 6.8 MB. For files larger than
>6.8
> > MB, it is necessary to use an XBAT. Excel will no longer open the file.
> > The error message states the file is not repairable and cannot extract
>data
> > from the file.
> >
> > We adapted the code to calculate the number of XBATs needed and made
> > appropriate changes to the self-description in the BATs. I've even gone
>so
> > far as to compare our generated file in HEX to an Excel XP file.
>Everything
> > looks the same for same-size files with the exception that we do not
> > compress our string table.
> >
> > At this point, OpenOffice will successfully open the file but Excel will
> > not. If we save the OpenOffice file as an Excel file, it does open
> > properly. Since we cannot ask our customers to do this, this is not an
> > viable solution.
> >
> > What potential pitfalls might we have fallen into? I'm thinking the
>problem
> > is related to our implementation of the XBAT, but I'm not sure since it
> > looks the same as an Excel XP file. I'd be more than willing to provide a
> > generated file to anyone who is able to help. Thanks in advance.
> >
> > Brandon Belvin
> > Information Systems Support, Inc.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: poi-user-help@jakarta.apache.org
> >
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: poi-user-help@jakarta.apache.org
Glen Stampoultzis
gstamp@iinet.net.au
http://members.iinet.net.au/~gstamp/glen/
Re: Problem generating a large file when following the
POIFSstandard
Posted by "Andrew C. Oliver" <ac...@apache.org>.
I'm not bragging. POIFS was originally written by me in a very primitive
form and called OLE2FS (but I never released code to the public so no
trademark violation), it was mainly to support HSSF and HSSF was mainly to
support the HSSF Serializer. Marc Johnson rewrote it, I contributed write
support, then he rewrote it 2 more times (at least) mainly to satiate needs
for SuperLink customers and increase HSSF efficiency. Marc then left the
project more or less though he occasionally does work for SuperLink. No one
else has actually worked on POIFS but me and Marc except for the occasional
minor patch.
The questions aren't about POIFS they are about the file format in general
and have nothing to do with USING POI (this is poi-user!). Next, the
poi-dev list which would seem to be more appropriate, isn't actually
appropriate because its for the contribution to developing POI.
This isn't the "free help with understanding Microsoft File formats" list,
it is the "POI users" list. If you are not asking questions about using POI
then you're off topic and I may actually unsubscribe you from the list (as
list moderator).
The poi-dev list would ALMOST be right except its not questions for
contribution or development of POI, its for development of a proprietary
fork of POI. It is not the place for that. We HAVE no list for that and
won't be founding one anytime soon.
I'm not arguing your right to help people fork the project for free (you can
do whatever you want), I'm arguing there is no right to post off-topic on
this list (which such requests are). As moderator of the list, I'm quite
serious and I do not feel a great deal of magnanimity for those creating
proprietary forks of the project regardless of their reasons. We've been
there, done that and I've got the T-Shirt.
Now move on.
-Andy
--
Andrew C. Oliver
http://www.superlinksoftware.com/poi.jsp
Custom enhancements and Commercial Implementation for Jakarta POI
http://jakarta.apache.org/poi
For Java and Excel, Got POI?
> From: "Kais Dukes" <kd...@kaisdukes.com>
> Reply-To: "POI Users List" <po...@jakarta.apache.org>
> Date: Thu, 26 Feb 2004 09:21:33 -0000
> To: "POI Users List" <po...@jakarta.apache.org>
> Subject: RE: Problem generating a large file when following the POIFSstandard
>
> Dear Andrew
>
> What sort of question is that? Did you not read anything I said. I TOTALLY
> RESPECT other people getting paid. Ive got nothing wrong with it. I just
> said that not everyone might want to. Whats the point of asking active
> committers if they want to get paid? My point was simply this, if poeple ask
> quesitons about POIFS *I* dont mind answering for free.
>
> You've decided to wrap up the guys question as a "properierty fork". I think
> answering questions about XBATs is fine -- it can only HELP this project,
> and HELP others understand POIFS.
>
> Also, do you think its a good idea to brag that "I'm the only left who
> understands POIFS"
>
> Apart from the fact that this is clearly untrue, thats not a good thing to
> go on about. In fact, its worrying. I would be more more proud to say "lots
> of people understand the core internals of my work". Instead of turning this
> guy away, why didnt we open a discussion on XBATs, and the implementation in
> POI? I would have been happy to discuss this.
>
> The more people the better. Your survey is both childish and a waste of
> time. If everyone repies "I want to get paid" then ... well ... they like
> getting paid. You missed my main point: SOME PEOPLE LIKE GIVING AWAY
> KNOWLEDGE FOR FREE. As glen rightly pointed out, the knowledge itself is
> more important than the code, and you dont have to be a comitter to have a
> good understanding of exactly how XLS and OLE2 compound documents work.
>
> -- Kais
>
> -----Original Message-----
> From: Andrew C. Oliver [mailto:acoliver@apache.org]
> Sent: 26 February 2004 03:28
> To: POI Users List
> Subject: Re: Problem generating a large file when following the
> POIFSstandard
>
>
> Pretty audacious, if merit on the project came in fluid form, Glen would
> have gallons.
>
> Lets save everyone a little time and trouble active committers please sound
> off. How many of you feel that you'd like to offer your services for free
> to someone developing a proprietary fork of this project?
>
> So far:
> Andrew C. Oliver - No.
>
> Glen Stampoultzis (expertise: HSSF, Graphing, Escher, Code generation,
> original advocate for unit tests, second oldest timer on the project) - No
>
> Rainer Klute
>
> Ryan Ackley
>
> Avik Sengupta
>
> Shawn Laubach
>
> Danny Mui
>
> Jason Height
>
> Assuming it isn't a majority, then you can contact that/those individuals
> directly instead of flooding the users list with questions that do not
> interest users of POI or potentially start a proprietary-poi-fork list
> somewhere else.
>
> -Andy
>
> --
> Andrew C. Oliver
> http://www.superlinksoftware.com/poi.jsp
> Custom enhancements and Commercial Implementation for Jakarta POI
>
> http://jakarta.apache.org/poi
> For Java and Excel, Got POI?
>
>> From: "Kais Dukes" <kd...@kaisdukes.com>
>> Reply-To: "POI Users List" <po...@jakarta.apache.org>
>> Date: Thu, 26 Feb 2004 02:46:25 -0000
>> To: "POI Users List" <po...@jakarta.apache.org>
>> Subject: RE: Problem generating a large file when following the
> POIFSstandard
>>
>> Hi Glen
>>
>> I respect what you are saying, but that doesnt mean that others should be
>> barred from offering free information if they want to, where it relates
>> directly to POI.
>>
>> -- Kais
>>
>> -----Original Message-----
>> From: Glen Stampoultzis [mailto:gstamp@iinet.net.au]
>> Sent: 26 February 2004 02:08
>> To: POI Users List
>> Subject: RE: Problem generating a large file when following the
>> POIFSstandard
>>
>>
>> You are right: the knowledge is the most valuable part. That's why we
>> prefer to be paid for that knowledge. Sometimes we give way that
> knowledge
>> for free but that's our choice and it's usually to further the cause of
> the
>> open source we are involved in and not to act as an unpaid employee.
>>
>> At 09:49 AM 26/02/2004, you wrote:
>>> Dear Andrew,
>>>
>>> The code itself is only one part of the project. Perhaps even more
> valuable
>>> than actual code is the documentation, and knowledge that poeple in this
>>> group have. Its important to discuss the actual standards themselves. The
>>> fact that Microsoft dont discuss these things, or make them public is
>>> EXACTLY the reason why this project is so valuable. Perhaps we shouldn't
> be
>>> so quick to turn away people asking questions about how POIFS persistance
>>> works. Answering these questions will help people understand the code,
> and
>>> find bugs, etc.
>>>
>>> The service is not free? Perhaps your time is not free, but if people are
>>> asking questions about how POIFS works, Im happy to provide a free
> service,
>>> wherever I can help. The quesiton was not C++ specific, it was merely
> about
>>> the POIFS standard, so I think it's perfectly acceptable.
>>>
>>> I dont think you should have to donate code in order to get an answer
> about
>>> how existing code works, even if you are deriving something from it,
>>> especially under an Apache license.
>>>
>>> -- Kais
>>>
>>> -----Original Message-----
>>> From: Andrew C. Oliver [mailto:acoliver@apache.org]
>>> Sent: 25 February 2004 22:40
>>> To: POI Users List
>>> Subject: Re: Problem generating a large file when following the
>>> POIFSstandard
>>>
>>>
>>> Unless you plan to donate the C++ code to the project, I'm not sure this
> is
>>> the appropriate place to ask your question. The source is open and free,
>>> the service (like helping you work on a proprietary language fork of POI)
>>> isn't.
>>> --
>>> Andrew C. Oliver
>>> http://www.superlinksoftware.com/poi.jsp
>>> Custom enhancements and Commercial Implementation for Jakarta POI
>>>
>>> http://jakarta.apache.org/poi
>>> For Java and Excel, Got POI?
>>>
>>> The views expressed in this email are those of the author and are almost
>>> definitely not shared by the Apache Software Foundation, its board or its
>>> general membership. In fact they probably most definitively disagree
> with
>>> everything espoused in the above email.
>>>
>>>> From: "Brandon Belvin" <bb...@iss-md.com>
>>>> Organization: Information Systems Support, Inc.
>>>> Reply-To: "POI Users List" <po...@jakarta.apache.org>
>>>> Date: Wed, 25 Feb 2004 16:04:00 -0600
>>>> To: "POI User Mailing List" <po...@jakarta.apache.org>
>>>> Subject: Problem generating a large file when following the POIFS
>> standard
>>>>
>>>> As an add-on to an existing database front-end application, we have
>>> allowed
>>>> the user to export their report's data to Excel. Following POIFS'
>>>> standards, we used C++ to write the Excel file. Upon testing, we
>>> discovered
>>>> our code works perfectly for files under 6.8 MB. For files larger than
>>> 6.8
>>>> MB, it is necessary to use an XBAT. Excel will no longer open the file.
>>>> The error message states the file is not repairable and cannot extract
>>> data
>>>> from the file.
>>>>
>>>> We adapted the code to calculate the number of XBATs needed and made
>>>> appropriate changes to the self-description in the BATs. I've even gone
>>> so
>>>> far as to compare our generated file in HEX to an Excel XP file.
>>> Everything
>>>> looks the same for same-size files with the exception that we do not
>>>> compress our string table.
>>>>
>>>> At this point, OpenOffice will successfully open the file but Excel will
>>>> not. If we save the OpenOffice file as an Excel file, it does open
>>>> properly. Since we cannot ask our customers to do this, this is not an
>>>> viable solution.
>>>>
>>>> What potential pitfalls might we have fallen into? I'm thinking the
>>> problem
>>>> is related to our implementation of the XBAT, but I'm not sure since it
>>>> looks the same as an Excel XP file. I'd be more than willing to provide
>> a
>>>> generated file to anyone who is able to help. Thanks in advance.
>>>>
>>>> Brandon Belvin
>>>> Information Systems Support, Inc.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>
>>
>> Glen Stampoultzis
>> gstamp@iinet.net.au
>> http://members.iinet.net.au/~gstamp/glen/
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
Re: Problem generating a large file when following the POIFSstandard
Posted by Jeff Blackwell <jb...@tenacityinc.com>.
Thanks for the warning... ;-)
Admittedly, this is an ugly case - unfortunately
necessary. Not that I'd hold anybody to it, but do
you have any ballpark ideas what constitutes a "Very
very long time" or how much an "Oppressive amount of
memory" is? Not trying to pin anyone down, I just
need to know if I should be looking at a non-Java
option, if this will be unworkable for the situation
that I need it for.
Thanks for the info, and for the obvious huge amount
of work that you've put in here.
-Jeff
--- "Andrew C. Oliver" <ac...@apache.org> wrote:
> It will load very very slowly. We also don't have
> support yet for
> "relative" rows and cells in HSSF. Originally when
> we wrote the first few
> iterations we thought "No one will generate a sheet
> that big". I'd like to
> see some testing done on the maximum valid XLS file,
> but I've not done it in
> awhile.
>
> Also to make that big of a file with POI 2.0 you'll
> find we'll consume an
> oppressive amount of memory. The garbage collector
> will run wild. 3.0
> should answer some of that once it gets under way.
> --
> Andrew C. Oliver
> http://www.superlinksoftware.com/poi.jsp
> Custom enhancements and Commercial Implementation
> for Jakarta POI
>
> http://jakarta.apache.org/poi
> For Java and Excel, Got POI?
>
> > From: Jeff Blackwell <jb...@tenacityinc.com>
> > Reply-To: "POI Users List"
> <po...@jakarta.apache.org>
> > Date: Thu, 26 Feb 2004 08:48:08 -0800 (PST)
> > To: POI Users List <po...@jakarta.apache.org>
> > Subject: RE: Problem generating a large file when
> following the POIFSstandard
> >
> > I haven't used POI very much. Rather new to the
> > POI/HSSF game, but been around Java for a while.
> Kais
> > seems to mention memory issues with writing large
> > files. Are there any gotcha's that I should be
> aware
> > of? Anything specific to avoid? I may have need
> to
> > go into the 20-40 mb range. I know, it's insane
> to
> > stuff that much rot in a sheet - but it honestly
> may
> > need to be done in this case. Again, I'm just
> > investigating, and need to know what to be aware
> of,
> > before jumping into it with both feet. Any light
> you
> > can shed on it would be much appreciated...
> >
> > Thanks in advance,
> >
> > Jeff
> >
> >
> > --- Kais Dukes <kd...@kaisdukes.com> wrote:
> >> Dear Michael
> >>
> >> I can offer my answers to these questions, with
> the
> >> disclaimer that these
> >> are based on my own observations.
> >>
> >> From my use of POIFS:
> >>
> >> (1) POIFS seems to have no trouble with large
> files,
> >> in terms of XBATs.
> >> Memory is another question.
> >> (2) You are right that XBAT is not the usual
> name,
> >> in Microsoft speak, and
> >> reading standard docs on OLE2, it is the DIF.
> >> (3) I have noticed these issues. In the
> >> implementations I have written for
> >> OLE2 structured storage, it appears that the
> >> Microsoft implementation will
> >> correctly read your OLE2 files regardless of how
> you
> >> distribute BAT or XBAT
> >> blocks (as long as there correctly pointed to).
> >> (4) The POIFS implementation doesn't seem to hard
> >> code sector sizes, these
> >> have been abstacted (e.g. contants). It may be
> the
> >> case that switching these
> >> would result in the code still working, although
> I
> >> have not tested.
> >> (5) Related to the issue of sector sizes, the
> next
> >> version of Windows
> >> (Longhorn) will have much better support sector
> >> sizes other than 512 bytes,
> >> so I would hope a future POIFS system to deal
> with
> >> this correctly.
> >>
> >> -----Original Message-----
> >> From: Michael Zalewski
> >> [mailto:zalewski@optonline.net]
> >> Sent: 26 February 2004 14:43
> >> To: POI Users List
> >> Subject: RE: Problem generating a large file when
> >> following the
> >> POIFSstandard
> >>
> >>
> >> I don't mean to add fuel to the fire. But I have
> >> some questions:
> >>
> >> 1. Brandon pointed out that a C++ implementation
> >> based on POIFS has trouble
> >> writing files larger than 6.8 MB, where XBATs
> must
> >> be created. Do we know
> >> that POIFS creates these large files correctly?
> Has
> >> anyone used HSSF to make
> >> a XLS larger than 7 MB? How much heap space was
> >> required?
> >>
> >> 2. For that matter, where does the term XBAT come
> >> from? I thought it was
> >> called DIF (Double Indirect FAT).
> >>
> >> 3. There are some other differences that I notice
> in
> >> the way POIFS handles
> >> things versus COMPOBJ.DLL. The most notable is
> that
> >> HSSF does not put
> >> something called the 'Storage Class ID' into the
> >> root entry or the OLE Doc
> >> file header. But Excel doesn't seem to care.
> POIFS
> >> has no support for
> >> 'CompObj' stream, which is required for embedding
> >> (and is used for Word and
> >> Project even when the files are not embedded -
> but
> >> again the application
> >> doesn't seem to care if the stream is present or
> not
> >> when the document is
> >> read from a file). POIFS cannot create a document
> >> stream of 0 length (it's a
> >> known bug). And finally, POIFS puts the directory
> >> structure in block 0
> >> whereas Office always puts the document stream
> into
> >> block 0. None of these
> >> differences should account for Brandon's problems
> (I
> >> don't think).
> >>
> >> 4. Does POIFS support any block size other than
> 512
> >> bytes? Not sure, but I
> >> think Microsoft Map uses a larger sector size.
> Since
> >> POIFSConstants.BIG_BLOCK_SIZE is statically
> defined
> >> (instead of interpreting
> >> the value at offset 30 from the beginning of the
> >> file. This value is (I
> >> guess) ignored by POIFS, but should be
> interpreted
> >> as the base 2 logarithm
> >> of the large sector size (so it should be 9 for
> 512
> >> bytes used by Microsoft
> >> Office, and I think 10 for 1024 bytes on some
> files
> >> produced by Microsoft
> >> Map).
> >>
> >> If Brandon's C++ derivative supports the variant
> >> block sizes, perhaps he
> >> could try using a 1024 block size. Doing this
> should
> >> allow you to go to over
> >> 12 MB without the need for XBATs (or DIFs). I
> would
> >> really like to know if
> >> that works - but I seriously wonder if he would
> >> report back anyway.
> >>
> >> Note: I am not at all certain about Microsoft
> Map.
> >> Just had reports of
> >> someone using POIFS to read that file, and
> >> everything was 'one block off'.
> >> When (if) I ever follow up on it, I will report
> >> results back to this list
> >> (unless I get viciously shouted down).
> >>
> >>
> >>
> >>
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
Re: Problem generating a large file when following the
POIFSstandard
Posted by "Andrew C. Oliver" <ac...@apache.org>.
It will load very very slowly. We also don't have support yet for
"relative" rows and cells in HSSF. Originally when we wrote the first few
iterations we thought "No one will generate a sheet that big". I'd like to
see some testing done on the maximum valid XLS file, but I've not done it in
awhile.
Also to make that big of a file with POI 2.0 you'll find we'll consume an
oppressive amount of memory. The garbage collector will run wild. 3.0
should answer some of that once it gets under way.
--
Andrew C. Oliver
http://www.superlinksoftware.com/poi.jsp
Custom enhancements and Commercial Implementation for Jakarta POI
http://jakarta.apache.org/poi
For Java and Excel, Got POI?
> From: Jeff Blackwell <jb...@tenacityinc.com>
> Reply-To: "POI Users List" <po...@jakarta.apache.org>
> Date: Thu, 26 Feb 2004 08:48:08 -0800 (PST)
> To: POI Users List <po...@jakarta.apache.org>
> Subject: RE: Problem generating a large file when following the POIFSstandard
>
> I haven't used POI very much. Rather new to the
> POI/HSSF game, but been around Java for a while. Kais
> seems to mention memory issues with writing large
> files. Are there any gotcha's that I should be aware
> of? Anything specific to avoid? I may have need to
> go into the 20-40 mb range. I know, it's insane to
> stuff that much rot in a sheet - but it honestly may
> need to be done in this case. Again, I'm just
> investigating, and need to know what to be aware of,
> before jumping into it with both feet. Any light you
> can shed on it would be much appreciated...
>
> Thanks in advance,
>
> Jeff
>
>
> --- Kais Dukes <kd...@kaisdukes.com> wrote:
>> Dear Michael
>>
>> I can offer my answers to these questions, with the
>> disclaimer that these
>> are based on my own observations.
>>
>> From my use of POIFS:
>>
>> (1) POIFS seems to have no trouble with large files,
>> in terms of XBATs.
>> Memory is another question.
>> (2) You are right that XBAT is not the usual name,
>> in Microsoft speak, and
>> reading standard docs on OLE2, it is the DIF.
>> (3) I have noticed these issues. In the
>> implementations I have written for
>> OLE2 structured storage, it appears that the
>> Microsoft implementation will
>> correctly read your OLE2 files regardless of how you
>> distribute BAT or XBAT
>> blocks (as long as there correctly pointed to).
>> (4) The POIFS implementation doesn't seem to hard
>> code sector sizes, these
>> have been abstacted (e.g. contants). It may be the
>> case that switching these
>> would result in the code still working, although I
>> have not tested.
>> (5) Related to the issue of sector sizes, the next
>> version of Windows
>> (Longhorn) will have much better support sector
>> sizes other than 512 bytes,
>> so I would hope a future POIFS system to deal with
>> this correctly.
>>
>> -----Original Message-----
>> From: Michael Zalewski
>> [mailto:zalewski@optonline.net]
>> Sent: 26 February 2004 14:43
>> To: POI Users List
>> Subject: RE: Problem generating a large file when
>> following the
>> POIFSstandard
>>
>>
>> I don't mean to add fuel to the fire. But I have
>> some questions:
>>
>> 1. Brandon pointed out that a C++ implementation
>> based on POIFS has trouble
>> writing files larger than 6.8 MB, where XBATs must
>> be created. Do we know
>> that POIFS creates these large files correctly? Has
>> anyone used HSSF to make
>> a XLS larger than 7 MB? How much heap space was
>> required?
>>
>> 2. For that matter, where does the term XBAT come
>> from? I thought it was
>> called DIF (Double Indirect FAT).
>>
>> 3. There are some other differences that I notice in
>> the way POIFS handles
>> things versus COMPOBJ.DLL. The most notable is that
>> HSSF does not put
>> something called the 'Storage Class ID' into the
>> root entry or the OLE Doc
>> file header. But Excel doesn't seem to care. POIFS
>> has no support for
>> 'CompObj' stream, which is required for embedding
>> (and is used for Word and
>> Project even when the files are not embedded - but
>> again the application
>> doesn't seem to care if the stream is present or not
>> when the document is
>> read from a file). POIFS cannot create a document
>> stream of 0 length (it's a
>> known bug). And finally, POIFS puts the directory
>> structure in block 0
>> whereas Office always puts the document stream into
>> block 0. None of these
>> differences should account for Brandon's problems (I
>> don't think).
>>
>> 4. Does POIFS support any block size other than 512
>> bytes? Not sure, but I
>> think Microsoft Map uses a larger sector size. Since
>> POIFSConstants.BIG_BLOCK_SIZE is statically defined
>> (instead of interpreting
>> the value at offset 30 from the beginning of the
>> file. This value is (I
>> guess) ignored by POIFS, but should be interpreted
>> as the base 2 logarithm
>> of the large sector size (so it should be 9 for 512
>> bytes used by Microsoft
>> Office, and I think 10 for 1024 bytes on some files
>> produced by Microsoft
>> Map).
>>
>> If Brandon's C++ derivative supports the variant
>> block sizes, perhaps he
>> could try using a 1024 block size. Doing this should
>> allow you to go to over
>> 12 MB without the need for XBATs (or DIFs). I would
>> really like to know if
>> that works - but I seriously wonder if he would
>> report back anyway.
>>
>> Note: I am not at all certain about Microsoft Map.
>> Just had reports of
>> someone using POIFS to read that file, and
>> everything was 'one block off'.
>> When (if) I ever follow up on it, I will report
>> results back to this list
>> (unless I get viciously shouted down).
>>
>>
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> poi-user-help@jakarta.apache.org
>>
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> poi-user-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
RE: Problem generating a large file when following the POIFSstandard
Posted by Jeff Blackwell <jb...@tenacityinc.com>.
I haven't used POI very much. Rather new to the
POI/HSSF game, but been around Java for a while. Kais
seems to mention memory issues with writing large
files. Are there any gotcha's that I should be aware
of? Anything specific to avoid? I may have need to
go into the 20-40 mb range. I know, it's insane to
stuff that much rot in a sheet - but it honestly may
need to be done in this case. Again, I'm just
investigating, and need to know what to be aware of,
before jumping into it with both feet. Any light you
can shed on it would be much appreciated...
Thanks in advance,
Jeff
--- Kais Dukes <kd...@kaisdukes.com> wrote:
> Dear Michael
>
> I can offer my answers to these questions, with the
> disclaimer that these
> are based on my own observations.
>
> From my use of POIFS:
>
> (1) POIFS seems to have no trouble with large files,
> in terms of XBATs.
> Memory is another question.
> (2) You are right that XBAT is not the usual name,
> in Microsoft speak, and
> reading standard docs on OLE2, it is the DIF.
> (3) I have noticed these issues. In the
> implementations I have written for
> OLE2 structured storage, it appears that the
> Microsoft implementation will
> correctly read your OLE2 files regardless of how you
> distribute BAT or XBAT
> blocks (as long as there correctly pointed to).
> (4) The POIFS implementation doesn't seem to hard
> code sector sizes, these
> have been abstacted (e.g. contants). It may be the
> case that switching these
> would result in the code still working, although I
> have not tested.
> (5) Related to the issue of sector sizes, the next
> version of Windows
> (Longhorn) will have much better support sector
> sizes other than 512 bytes,
> so I would hope a future POIFS system to deal with
> this correctly.
>
> -----Original Message-----
> From: Michael Zalewski
> [mailto:zalewski@optonline.net]
> Sent: 26 February 2004 14:43
> To: POI Users List
> Subject: RE: Problem generating a large file when
> following the
> POIFSstandard
>
>
> I don't mean to add fuel to the fire. But I have
> some questions:
>
> 1. Brandon pointed out that a C++ implementation
> based on POIFS has trouble
> writing files larger than 6.8 MB, where XBATs must
> be created. Do we know
> that POIFS creates these large files correctly? Has
> anyone used HSSF to make
> a XLS larger than 7 MB? How much heap space was
> required?
>
> 2. For that matter, where does the term XBAT come
> from? I thought it was
> called DIF (Double Indirect FAT).
>
> 3. There are some other differences that I notice in
> the way POIFS handles
> things versus COMPOBJ.DLL. The most notable is that
> HSSF does not put
> something called the 'Storage Class ID' into the
> root entry or the OLE Doc
> file header. But Excel doesn't seem to care. POIFS
> has no support for
> 'CompObj' stream, which is required for embedding
> (and is used for Word and
> Project even when the files are not embedded - but
> again the application
> doesn't seem to care if the stream is present or not
> when the document is
> read from a file). POIFS cannot create a document
> stream of 0 length (it's a
> known bug). And finally, POIFS puts the directory
> structure in block 0
> whereas Office always puts the document stream into
> block 0. None of these
> differences should account for Brandon's problems (I
> don't think).
>
> 4. Does POIFS support any block size other than 512
> bytes? Not sure, but I
> think Microsoft Map uses a larger sector size. Since
> POIFSConstants.BIG_BLOCK_SIZE is statically defined
> (instead of interpreting
> the value at offset 30 from the beginning of the
> file. This value is (I
> guess) ignored by POIFS, but should be interpreted
> as the base 2 logarithm
> of the large sector size (so it should be 9 for 512
> bytes used by Microsoft
> Office, and I think 10 for 1024 bytes on some files
> produced by Microsoft
> Map).
>
> If Brandon's C++ derivative supports the variant
> block sizes, perhaps he
> could try using a 1024 block size. Doing this should
> allow you to go to over
> 12 MB without the need for XBATs (or DIFs). I would
> really like to know if
> that works - but I seriously wonder if he would
> report back anyway.
>
> Note: I am not at all certain about Microsoft Map.
> Just had reports of
> someone using POIFS to read that file, and
> everything was 'one block off'.
> When (if) I ever follow up on it, I will report
> results back to this list
> (unless I get viciously shouted down).
>
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> poi-user-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> poi-user-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
RE: Problem generating a large file when following the POIFSstandard
Posted by Kais Dukes <kd...@kaisdukes.com>.
Dear Michael
I can offer my answers to these questions, with the disclaimer that these
are based on my own observations.
>From my use of POIFS:
(1) POIFS seems to have no trouble with large files, in terms of XBATs.
Memory is another question.
(2) You are right that XBAT is not the usual name, in Microsoft speak, and
reading standard docs on OLE2, it is the DIF.
(3) I have noticed these issues. In the implementations I have written for
OLE2 structured storage, it appears that the Microsoft implementation will
correctly read your OLE2 files regardless of how you distribute BAT or XBAT
blocks (as long as there correctly pointed to).
(4) The POIFS implementation doesn't seem to hard code sector sizes, these
have been abstacted (e.g. contants). It may be the case that switching these
would result in the code still working, although I have not tested.
(5) Related to the issue of sector sizes, the next version of Windows
(Longhorn) will have much better support sector sizes other than 512 bytes,
so I would hope a future POIFS system to deal with this correctly.
-----Original Message-----
From: Michael Zalewski [mailto:zalewski@optonline.net]
Sent: 26 February 2004 14:43
To: POI Users List
Subject: RE: Problem generating a large file when following the
POIFSstandard
I don't mean to add fuel to the fire. But I have some questions:
1. Brandon pointed out that a C++ implementation based on POIFS has trouble
writing files larger than 6.8 MB, where XBATs must be created. Do we know
that POIFS creates these large files correctly? Has anyone used HSSF to make
a XLS larger than 7 MB? How much heap space was required?
2. For that matter, where does the term XBAT come from? I thought it was
called DIF (Double Indirect FAT).
3. There are some other differences that I notice in the way POIFS handles
things versus COMPOBJ.DLL. The most notable is that HSSF does not put
something called the 'Storage Class ID' into the root entry or the OLE Doc
file header. But Excel doesn't seem to care. POIFS has no support for
'CompObj' stream, which is required for embedding (and is used for Word and
Project even when the files are not embedded - but again the application
doesn't seem to care if the stream is present or not when the document is
read from a file). POIFS cannot create a document stream of 0 length (it's a
known bug). And finally, POIFS puts the directory structure in block 0
whereas Office always puts the document stream into block 0. None of these
differences should account for Brandon's problems (I don't think).
4. Does POIFS support any block size other than 512 bytes? Not sure, but I
think Microsoft Map uses a larger sector size. Since
POIFSConstants.BIG_BLOCK_SIZE is statically defined (instead of interpreting
the value at offset 30 from the beginning of the file. This value is (I
guess) ignored by POIFS, but should be interpreted as the base 2 logarithm
of the large sector size (so it should be 9 for 512 bytes used by Microsoft
Office, and I think 10 for 1024 bytes on some files produced by Microsoft
Map).
If Brandon's C++ derivative supports the variant block sizes, perhaps he
could try using a 1024 block size. Doing this should allow you to go to over
12 MB without the need for XBATs (or DIFs). I would really like to know if
that works - but I seriously wonder if he would report back anyway.
Note: I am not at all certain about Microsoft Map. Just had reports of
someone using POIFS to read that file, and everything was 'one block off'.
When (if) I ever follow up on it, I will report results back to this list
(unless I get viciously shouted down).
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
Re: Problem generating a large file when following the
POIFSstandard
Posted by "Andrew C. Oliver" <ac...@apache.org>.
If this is a discussion towards the development of POI and not on usage it
should move to POI-dev.
--
Andrew C. Oliver
http://www.superlinksoftware.com/poi.jsp
Custom enhancements and Commercial Implementation for Jakarta POI
http://jakarta.apache.org/poi
For Java and Excel, Got POI?
> From: Michael Zalewski <za...@optonline.net>
> Reply-To: "POI Users List" <po...@jakarta.apache.org>
> Date: Thu, 26 Feb 2004 09:43:18 -0500
> To: POI Users List <po...@jakarta.apache.org>
> Subject: RE: Problem generating a large file when following the POIFSstandard
>
> I don't mean to add fuel to the fire. But I have some questions:
>
> 1. Brandon pointed out that a C++ implementation based on POIFS has trouble
> writing files larger than 6.8 MB, where XBATs must be created. Do we know
> that POIFS creates these large files correctly? Has anyone used HSSF to make
> a XLS larger than 7 MB? How much heap space was required?
>
> 2. For that matter, where does the term XBAT come from? I thought it was
> called DIF (Double Indirect FAT).
>
> 3. There are some other differences that I notice in the way POIFS handles
> things versus COMPOBJ.DLL. The most notable is that HSSF does not put
> something called the 'Storage Class ID' into the root entry or the OLE Doc
> file header. But Excel doesn't seem to care. POIFS has no support for
> 'CompObj' stream, which is required for embedding (and is used for Word and
> Project even when the files are not embedded - but again the application
> doesn't seem to care if the stream is present or not when the document is
> read from a file). POIFS cannot create a document stream of 0 length (it's a
> known bug). And finally, POIFS puts the directory structure in block 0
> whereas Office always puts the document stream into block 0. None of these
> differences should account for Brandon's problems (I don't think).
>
> 4. Does POIFS support any block size other than 512 bytes? Not sure, but I
> think Microsoft Map uses a larger sector size. Since
> POIFSConstants.BIG_BLOCK_SIZE is statically defined (instead of interpreting
> the value at offset 30 from the beginning of the file. This value is (I
> guess) ignored by POIFS, but should be interpreted as the base 2 logarithm
> of the large sector size (so it should be 9 for 512 bytes used by Microsoft
> Office, and I think 10 for 1024 bytes on some files produced by Microsoft
> Map).
>
> If Brandon's C++ derivative supports the variant block sizes, perhaps he
> could try using a 1024 block size. Doing this should allow you to go to over
> 12 MB without the need for XBATs (or DIFs). I would really like to know if
> that works - but I seriously wonder if he would report back anyway.
>
> Note: I am not at all certain about Microsoft Map. Just had reports of
> someone using POIFS to read that file, and everything was 'one block off'.
> When (if) I ever follow up on it, I will report results back to this list
> (unless I get viciously shouted down).
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
RE: Problem generating a large file when following the POIFSstandard
Posted by Michael Zalewski <za...@optonline.net>.
I don't mean to add fuel to the fire. But I have some questions:
1. Brandon pointed out that a C++ implementation based on POIFS has trouble
writing files larger than 6.8 MB, where XBATs must be created. Do we know
that POIFS creates these large files correctly? Has anyone used HSSF to make
a XLS larger than 7 MB? How much heap space was required?
2. For that matter, where does the term XBAT come from? I thought it was
called DIF (Double Indirect FAT).
3. There are some other differences that I notice in the way POIFS handles
things versus COMPOBJ.DLL. The most notable is that HSSF does not put
something called the 'Storage Class ID' into the root entry or the OLE Doc
file header. But Excel doesn't seem to care. POIFS has no support for
'CompObj' stream, which is required for embedding (and is used for Word and
Project even when the files are not embedded - but again the application
doesn't seem to care if the stream is present or not when the document is
read from a file). POIFS cannot create a document stream of 0 length (it's a
known bug). And finally, POIFS puts the directory structure in block 0
whereas Office always puts the document stream into block 0. None of these
differences should account for Brandon's problems (I don't think).
4. Does POIFS support any block size other than 512 bytes? Not sure, but I
think Microsoft Map uses a larger sector size. Since
POIFSConstants.BIG_BLOCK_SIZE is statically defined (instead of interpreting
the value at offset 30 from the beginning of the file. This value is (I
guess) ignored by POIFS, but should be interpreted as the base 2 logarithm
of the large sector size (so it should be 9 for 512 bytes used by Microsoft
Office, and I think 10 for 1024 bytes on some files produced by Microsoft
Map).
If Brandon's C++ derivative supports the variant block sizes, perhaps he
could try using a 1024 block size. Doing this should allow you to go to over
12 MB without the need for XBATs (or DIFs). I would really like to know if
that works - but I seriously wonder if he would report back anyway.
Note: I am not at all certain about Microsoft Map. Just had reports of
someone using POIFS to read that file, and everything was 'one block off'.
When (if) I ever follow up on it, I will report results back to this list
(unless I get viciously shouted down).
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
RE: Problem generating a large file when following the POIFSstandard
Posted by Kais Dukes <kd...@kaisdukes.com>.
Dear Andrew
What sort of question is that? Did you not read anything I said. I TOTALLY
RESPECT other people getting paid. Ive got nothing wrong with it. I just
said that not everyone might want to. Whats the point of asking active
committers if they want to get paid? My point was simply this, if poeple ask
quesitons about POIFS *I* dont mind answering for free.
You've decided to wrap up the guys question as a "properierty fork". I think
answering questions about XBATs is fine -- it can only HELP this project,
and HELP others understand POIFS.
Also, do you think its a good idea to brag that "I'm the only left who
understands POIFS"
Apart from the fact that this is clearly untrue, thats not a good thing to
go on about. In fact, its worrying. I would be more more proud to say "lots
of people understand the core internals of my work". Instead of turning this
guy away, why didnt we open a discussion on XBATs, and the implementation in
POI? I would have been happy to discuss this.
The more people the better. Your survey is both childish and a waste of
time. If everyone repies "I want to get paid" then ... well ... they like
getting paid. You missed my main point: SOME PEOPLE LIKE GIVING AWAY
KNOWLEDGE FOR FREE. As glen rightly pointed out, the knowledge itself is
more important than the code, and you dont have to be a comitter to have a
good understanding of exactly how XLS and OLE2 compound documents work.
-- Kais
-----Original Message-----
From: Andrew C. Oliver [mailto:acoliver@apache.org]
Sent: 26 February 2004 03:28
To: POI Users List
Subject: Re: Problem generating a large file when following the
POIFSstandard
Pretty audacious, if merit on the project came in fluid form, Glen would
have gallons.
Lets save everyone a little time and trouble active committers please sound
off. How many of you feel that you'd like to offer your services for free
to someone developing a proprietary fork of this project?
So far:
Andrew C. Oliver - No.
Glen Stampoultzis (expertise: HSSF, Graphing, Escher, Code generation,
original advocate for unit tests, second oldest timer on the project) - No
Rainer Klute
Ryan Ackley
Avik Sengupta
Shawn Laubach
Danny Mui
Jason Height
Assuming it isn't a majority, then you can contact that/those individuals
directly instead of flooding the users list with questions that do not
interest users of POI or potentially start a proprietary-poi-fork list
somewhere else.
-Andy
--
Andrew C. Oliver
http://www.superlinksoftware.com/poi.jsp
Custom enhancements and Commercial Implementation for Jakarta POI
http://jakarta.apache.org/poi
For Java and Excel, Got POI?
> From: "Kais Dukes" <kd...@kaisdukes.com>
> Reply-To: "POI Users List" <po...@jakarta.apache.org>
> Date: Thu, 26 Feb 2004 02:46:25 -0000
> To: "POI Users List" <po...@jakarta.apache.org>
> Subject: RE: Problem generating a large file when following the
POIFSstandard
>
> Hi Glen
>
> I respect what you are saying, but that doesnt mean that others should be
> barred from offering free information if they want to, where it relates
> directly to POI.
>
> -- Kais
>
> -----Original Message-----
> From: Glen Stampoultzis [mailto:gstamp@iinet.net.au]
> Sent: 26 February 2004 02:08
> To: POI Users List
> Subject: RE: Problem generating a large file when following the
> POIFSstandard
>
>
> You are right: the knowledge is the most valuable part. That's why we
> prefer to be paid for that knowledge. Sometimes we give way that
knowledge
> for free but that's our choice and it's usually to further the cause of
the
> open source we are involved in and not to act as an unpaid employee.
>
> At 09:49 AM 26/02/2004, you wrote:
>> Dear Andrew,
>>
>> The code itself is only one part of the project. Perhaps even more
valuable
>> than actual code is the documentation, and knowledge that poeple in this
>> group have. Its important to discuss the actual standards themselves. The
>> fact that Microsoft dont discuss these things, or make them public is
>> EXACTLY the reason why this project is so valuable. Perhaps we shouldn't
be
>> so quick to turn away people asking questions about how POIFS persistance
>> works. Answering these questions will help people understand the code,
and
>> find bugs, etc.
>>
>> The service is not free? Perhaps your time is not free, but if people are
>> asking questions about how POIFS works, Im happy to provide a free
service,
>> wherever I can help. The quesiton was not C++ specific, it was merely
about
>> the POIFS standard, so I think it's perfectly acceptable.
>>
>> I dont think you should have to donate code in order to get an answer
about
>> how existing code works, even if you are deriving something from it,
>> especially under an Apache license.
>>
>> -- Kais
>>
>> -----Original Message-----
>> From: Andrew C. Oliver [mailto:acoliver@apache.org]
>> Sent: 25 February 2004 22:40
>> To: POI Users List
>> Subject: Re: Problem generating a large file when following the
>> POIFSstandard
>>
>>
>> Unless you plan to donate the C++ code to the project, I'm not sure this
is
>> the appropriate place to ask your question. The source is open and free,
>> the service (like helping you work on a proprietary language fork of POI)
>> isn't.
>> --
>> Andrew C. Oliver
>> http://www.superlinksoftware.com/poi.jsp
>> Custom enhancements and Commercial Implementation for Jakarta POI
>>
>> http://jakarta.apache.org/poi
>> For Java and Excel, Got POI?
>>
>> The views expressed in this email are those of the author and are almost
>> definitely not shared by the Apache Software Foundation, its board or its
>> general membership. In fact they probably most definitively disagree
with
>> everything espoused in the above email.
>>
>>> From: "Brandon Belvin" <bb...@iss-md.com>
>>> Organization: Information Systems Support, Inc.
>>> Reply-To: "POI Users List" <po...@jakarta.apache.org>
>>> Date: Wed, 25 Feb 2004 16:04:00 -0600
>>> To: "POI User Mailing List" <po...@jakarta.apache.org>
>>> Subject: Problem generating a large file when following the POIFS
> standard
>>>
>>> As an add-on to an existing database front-end application, we have
>> allowed
>>> the user to export their report's data to Excel. Following POIFS'
>>> standards, we used C++ to write the Excel file. Upon testing, we
>> discovered
>>> our code works perfectly for files under 6.8 MB. For files larger than
>> 6.8
>>> MB, it is necessary to use an XBAT. Excel will no longer open the file.
>>> The error message states the file is not repairable and cannot extract
>> data
>>> from the file.
>>>
>>> We adapted the code to calculate the number of XBATs needed and made
>>> appropriate changes to the self-description in the BATs. I've even gone
>> so
>>> far as to compare our generated file in HEX to an Excel XP file.
>> Everything
>>> looks the same for same-size files with the exception that we do not
>>> compress our string table.
>>>
>>> At this point, OpenOffice will successfully open the file but Excel will
>>> not. If we save the OpenOffice file as an Excel file, it does open
>>> properly. Since we cannot ask our customers to do this, this is not an
>>> viable solution.
>>>
>>> What potential pitfalls might we have fallen into? I'm thinking the
>> problem
>>> is related to our implementation of the XBAT, but I'm not sure since it
>>> looks the same as an Excel XP file. I'd be more than willing to provide
> a
>>> generated file to anyone who is able to help. Thanks in advance.
>>>
>>> Brandon Belvin
>>> Information Systems Support, Inc.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
>
> Glen Stampoultzis
> gstamp@iinet.net.au
> http://members.iinet.net.au/~gstamp/glen/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
Re: Problem generating a large file when following the
POIFSstandard
Posted by "Andrew C. Oliver" <ac...@apache.org>.
Pretty audacious, if merit on the project came in fluid form, Glen would
have gallons.
Lets save everyone a little time and trouble active committers please sound
off. How many of you feel that you'd like to offer your services for free
to someone developing a proprietary fork of this project?
So far:
Andrew C. Oliver - No.
Glen Stampoultzis (expertise: HSSF, Graphing, Escher, Code generation,
original advocate for unit tests, second oldest timer on the project) - No
Rainer Klute
Ryan Ackley
Avik Sengupta
Shawn Laubach
Danny Mui
Jason Height
Assuming it isn't a majority, then you can contact that/those individuals
directly instead of flooding the users list with questions that do not
interest users of POI or potentially start a proprietary-poi-fork list
somewhere else.
-Andy
--
Andrew C. Oliver
http://www.superlinksoftware.com/poi.jsp
Custom enhancements and Commercial Implementation for Jakarta POI
http://jakarta.apache.org/poi
For Java and Excel, Got POI?
> From: "Kais Dukes" <kd...@kaisdukes.com>
> Reply-To: "POI Users List" <po...@jakarta.apache.org>
> Date: Thu, 26 Feb 2004 02:46:25 -0000
> To: "POI Users List" <po...@jakarta.apache.org>
> Subject: RE: Problem generating a large file when following the POIFSstandard
>
> Hi Glen
>
> I respect what you are saying, but that doesnt mean that others should be
> barred from offering free information if they want to, where it relates
> directly to POI.
>
> -- Kais
>
> -----Original Message-----
> From: Glen Stampoultzis [mailto:gstamp@iinet.net.au]
> Sent: 26 February 2004 02:08
> To: POI Users List
> Subject: RE: Problem generating a large file when following the
> POIFSstandard
>
>
> You are right: the knowledge is the most valuable part. That's why we
> prefer to be paid for that knowledge. Sometimes we give way that knowledge
> for free but that's our choice and it's usually to further the cause of the
> open source we are involved in and not to act as an unpaid employee.
>
> At 09:49 AM 26/02/2004, you wrote:
>> Dear Andrew,
>>
>> The code itself is only one part of the project. Perhaps even more valuable
>> than actual code is the documentation, and knowledge that poeple in this
>> group have. Its important to discuss the actual standards themselves. The
>> fact that Microsoft dont discuss these things, or make them public is
>> EXACTLY the reason why this project is so valuable. Perhaps we shouldn't be
>> so quick to turn away people asking questions about how POIFS persistance
>> works. Answering these questions will help people understand the code, and
>> find bugs, etc.
>>
>> The service is not free? Perhaps your time is not free, but if people are
>> asking questions about how POIFS works, Im happy to provide a free service,
>> wherever I can help. The quesiton was not C++ specific, it was merely about
>> the POIFS standard, so I think it's perfectly acceptable.
>>
>> I dont think you should have to donate code in order to get an answer about
>> how existing code works, even if you are deriving something from it,
>> especially under an Apache license.
>>
>> -- Kais
>>
>> -----Original Message-----
>> From: Andrew C. Oliver [mailto:acoliver@apache.org]
>> Sent: 25 February 2004 22:40
>> To: POI Users List
>> Subject: Re: Problem generating a large file when following the
>> POIFSstandard
>>
>>
>> Unless you plan to donate the C++ code to the project, I'm not sure this is
>> the appropriate place to ask your question. The source is open and free,
>> the service (like helping you work on a proprietary language fork of POI)
>> isn't.
>> --
>> Andrew C. Oliver
>> http://www.superlinksoftware.com/poi.jsp
>> Custom enhancements and Commercial Implementation for Jakarta POI
>>
>> http://jakarta.apache.org/poi
>> For Java and Excel, Got POI?
>>
>> The views expressed in this email are those of the author and are almost
>> definitely not shared by the Apache Software Foundation, its board or its
>> general membership. In fact they probably most definitively disagree with
>> everything espoused in the above email.
>>
>>> From: "Brandon Belvin" <bb...@iss-md.com>
>>> Organization: Information Systems Support, Inc.
>>> Reply-To: "POI Users List" <po...@jakarta.apache.org>
>>> Date: Wed, 25 Feb 2004 16:04:00 -0600
>>> To: "POI User Mailing List" <po...@jakarta.apache.org>
>>> Subject: Problem generating a large file when following the POIFS
> standard
>>>
>>> As an add-on to an existing database front-end application, we have
>> allowed
>>> the user to export their report's data to Excel. Following POIFS'
>>> standards, we used C++ to write the Excel file. Upon testing, we
>> discovered
>>> our code works perfectly for files under 6.8 MB. For files larger than
>> 6.8
>>> MB, it is necessary to use an XBAT. Excel will no longer open the file.
>>> The error message states the file is not repairable and cannot extract
>> data
>>> from the file.
>>>
>>> We adapted the code to calculate the number of XBATs needed and made
>>> appropriate changes to the self-description in the BATs. I've even gone
>> so
>>> far as to compare our generated file in HEX to an Excel XP file.
>> Everything
>>> looks the same for same-size files with the exception that we do not
>>> compress our string table.
>>>
>>> At this point, OpenOffice will successfully open the file but Excel will
>>> not. If we save the OpenOffice file as an Excel file, it does open
>>> properly. Since we cannot ask our customers to do this, this is not an
>>> viable solution.
>>>
>>> What potential pitfalls might we have fallen into? I'm thinking the
>> problem
>>> is related to our implementation of the XBAT, but I'm not sure since it
>>> looks the same as an Excel XP file. I'd be more than willing to provide
> a
>>> generated file to anyone who is able to help. Thanks in advance.
>>>
>>> Brandon Belvin
>>> Information Systems Support, Inc.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
>
> Glen Stampoultzis
> gstamp@iinet.net.au
> http://members.iinet.net.au/~gstamp/glen/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org
RE: Problem generating a large file when following the POIFSstandard
Posted by Kais Dukes <kd...@kaisdukes.com>.
Hi Glen
I respect what you are saying, but that doesnt mean that others should be
barred from offering free information if they want to, where it relates
directly to POI.
-- Kais
-----Original Message-----
From: Glen Stampoultzis [mailto:gstamp@iinet.net.au]
Sent: 26 February 2004 02:08
To: POI Users List
Subject: RE: Problem generating a large file when following the
POIFSstandard
You are right: the knowledge is the most valuable part. That's why we
prefer to be paid for that knowledge. Sometimes we give way that knowledge
for free but that's our choice and it's usually to further the cause of the
open source we are involved in and not to act as an unpaid employee.
At 09:49 AM 26/02/2004, you wrote:
>Dear Andrew,
>
>The code itself is only one part of the project. Perhaps even more valuable
>than actual code is the documentation, and knowledge that poeple in this
>group have. Its important to discuss the actual standards themselves. The
>fact that Microsoft dont discuss these things, or make them public is
>EXACTLY the reason why this project is so valuable. Perhaps we shouldn't be
>so quick to turn away people asking questions about how POIFS persistance
>works. Answering these questions will help people understand the code, and
>find bugs, etc.
>
>The service is not free? Perhaps your time is not free, but if people are
>asking questions about how POIFS works, Im happy to provide a free service,
>wherever I can help. The quesiton was not C++ specific, it was merely about
>the POIFS standard, so I think it's perfectly acceptable.
>
>I dont think you should have to donate code in order to get an answer about
>how existing code works, even if you are deriving something from it,
>especially under an Apache license.
>
>-- Kais
>
>-----Original Message-----
>From: Andrew C. Oliver [mailto:acoliver@apache.org]
>Sent: 25 February 2004 22:40
>To: POI Users List
>Subject: Re: Problem generating a large file when following the
>POIFSstandard
>
>
>Unless you plan to donate the C++ code to the project, I'm not sure this is
>the appropriate place to ask your question. The source is open and free,
>the service (like helping you work on a proprietary language fork of POI)
>isn't.
>--
>Andrew C. Oliver
>http://www.superlinksoftware.com/poi.jsp
>Custom enhancements and Commercial Implementation for Jakarta POI
>
>http://jakarta.apache.org/poi
>For Java and Excel, Got POI?
>
>The views expressed in this email are those of the author and are almost
>definitely not shared by the Apache Software Foundation, its board or its
>general membership. In fact they probably most definitively disagree with
>everything espoused in the above email.
>
> > From: "Brandon Belvin" <bb...@iss-md.com>
> > Organization: Information Systems Support, Inc.
> > Reply-To: "POI Users List" <po...@jakarta.apache.org>
> > Date: Wed, 25 Feb 2004 16:04:00 -0600
> > To: "POI User Mailing List" <po...@jakarta.apache.org>
> > Subject: Problem generating a large file when following the POIFS
standard
> >
> > As an add-on to an existing database front-end application, we have
>allowed
> > the user to export their report's data to Excel. Following POIFS'
> > standards, we used C++ to write the Excel file. Upon testing, we
>discovered
> > our code works perfectly for files under 6.8 MB. For files larger than
>6.8
> > MB, it is necessary to use an XBAT. Excel will no longer open the file.
> > The error message states the file is not repairable and cannot extract
>data
> > from the file.
> >
> > We adapted the code to calculate the number of XBATs needed and made
> > appropriate changes to the self-description in the BATs. I've even gone
>so
> > far as to compare our generated file in HEX to an Excel XP file.
>Everything
> > looks the same for same-size files with the exception that we do not
> > compress our string table.
> >
> > At this point, OpenOffice will successfully open the file but Excel will
> > not. If we save the OpenOffice file as an Excel file, it does open
> > properly. Since we cannot ask our customers to do this, this is not an
> > viable solution.
> >
> > What potential pitfalls might we have fallen into? I'm thinking the
>problem
> > is related to our implementation of the XBAT, but I'm not sure since it
> > looks the same as an Excel XP file. I'd be more than willing to provide
a
> > generated file to anyone who is able to help. Thanks in advance.
> >
> > Brandon Belvin
> > Information Systems Support, Inc.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: poi-user-help@jakarta.apache.org
> >
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: poi-user-help@jakarta.apache.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: poi-user-help@jakarta.apache.org
Glen Stampoultzis
gstamp@iinet.net.au
http://members.iinet.net.au/~gstamp/glen/
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: poi-user-help@jakarta.apache.org