Memory management strategy

[Patricia Shanahan <pa...@acm.org>]

Currently, AOO uses a lot of finite size arrays. Ideally, overflows will 
be detected and lead to an error message, but that still prevents 
processing of the user's documents.

Computer memories and swap spaces are still increasing in size, so the 
array size limits are more likely to be the limiting factor on document 
sizes.

I can see two main strategies, but I'm open to additional suggestions.

1) Keep the finite size arrays, but periodically review them to see if 
the sizes should be increased.

2) Use automatically extending STL structures. If a document is too big 
or uses too many instances of some array element, the symptom will be 
the process as a whole running out of memory. We would have to make sure 
all memory allocation failures are handled smoothly.

This somewhat ties in with exception handling strategy. As we turn what 
used to be silent out-of-bounds conditions into exceptions, it is 
important to avoid arbitrary catching and hiding of all exceptions.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org

Re: Memory management strategy

[Damjan Jovanovic <da...@apache.org>]

A related problem to consider is the generally poor runtime environment
support for using C++ between modules. On Windows, using C++ between
different DLLs requires they were built by the same MSVC version and used
the same CRT runtime settings, and I am finding that for the main/jvmaccess
module to compile and link and be linked to from other modules in gbuild,
class typeinfos must be hidden and symbols for methods selectively
exported, on a case-by-case basis 🙈.

Exceptions in UNO components must be declared all the way from the IDL
file, so that the UNO bridge can catch them and re-create and rethrow them
on the caller's side (who is possibly calling that method using an RPC from
another process/machine). You can't just throw whatever you want, which the
C++ language itself allows. Changes to IDL files break compatibility with
clients using older versions, and are generally avoided, you can see many
examples of how an UNO interface/service got deprecated and a new one got
added instead.

Damjan


On Sun, Mar 25, 2018 at 5:18 PM, Peter Kovacs <Pe...@apache.org> wrote:

> Documents that are to large for the memory available will also fail to
> load in  the 1st approach.
> To add array review mechanism is in my opinion only a source of errors.
> I think STL should be used.
>
> +1 for not hiding the exceptions. Maybe we could catch exception on
> document level quickly and process the directly?
>
>
> On 25.03.2018 16:21, Patricia Shanahan wrote:
>
>> Currently, AOO uses a lot of finite size arrays. Ideally, overflows will
>> be detected and lead to an error message, but that still prevents
>> processing of the user's documents.
>>
>> Computer memories and swap spaces are still increasing in size, so the
>> array size limits are more likely to be the limiting factor on document
>> sizes.
>>
>> I can see two main strategies, but I'm open to additional suggestions.
>>
>> 1) Keep the finite size arrays, but periodically review them to see if
>> the sizes should be increased.
>>
>> 2) Use automatically extending STL structures. If a document is too big
>> or uses too many instances of some array element, the symptom will be the
>> process as a whole running out of memory. We would have to make sure all
>> memory allocation failures are handled smoothly.
>>
>> This somewhat ties in with exception handling strategy. As we turn what
>> used to be silent out-of-bounds conditions into exceptions, it is important
>> to avoid arbitrary catching and hiding of all exceptions.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>

Re: Memory management strategy

[Peter Kovacs <Pe...@Apache.org>]

Documents that are to large for the memory available will also fail to 
load in  the 1st approach.
To add array review mechanism is in my opinion only a source of errors.
I think STL should be used.

+1 for not hiding the exceptions. Maybe we could catch exception on 
document level quickly and process the directly?

On 25.03.2018 16:21, Patricia Shanahan wrote:
> Currently, AOO uses a lot of finite size arrays. Ideally, overflows 
> will be detected and lead to an error message, but that still prevents 
> processing of the user's documents.
>
> Computer memories and swap spaces are still increasing in size, so the 
> array size limits are more likely to be the limiting factor on 
> document sizes.
>
> I can see two main strategies, but I'm open to additional suggestions.
>
> 1) Keep the finite size arrays, but periodically review them to see if 
> the sizes should be increased.
>
> 2) Use automatically extending STL structures. If a document is too 
> big or uses too many instances of some array element, the symptom will 
> be the process as a whole running out of memory. We would have to make 
> sure all memory allocation failures are handled smoothly.
>
> This somewhat ties in with exception handling strategy. As we turn 
> what used to be silent out-of-bounds conditions into exceptions, it is 
> important to avoid arbitrary catching and hiding of all exceptions.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org