You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@corinthia.apache.org by Gabriela Gibson <ga...@gmail.com> on 2015/02/26 21:59:46 UTC
More on the xmalloc project
Hi Peter, (and everyone else!)
I read your JIRA comment, and that's an interesting thing with the missing
headerfiles. My build report[1] was OK so, maybe different compilers take
headers in a different order?
Regards the clang build, should/could we have this as a regular build
option in the CMake file?
Another thing to notice was that my test had quite a lot of malloc 0 calls,
so whatever is run there, maybe it could do with a check, or perhaps
xmalloc should check for if (!size) ... ?
The html file I downloaded and tested to get all those zeros was here:
http://stackoverflow.com/questions/8880603/loop-through-array-of-strings-in-bash-script
cmd I used to test was:
./bin/dfutil -normalize
~/cor2/incubator-corinthia/loop-through-array-of-strings-in-bash-script.html
Towards the end, there's a heap of 0 allocations.
G
Ps.: Currently trying to figure out the best way of testing the mallocs in
the WrapperTest.c file. Will holler when/if I get stuck.
[1] My build report:
make
[ 5%] Built target odf
[ 7%] Built target api
[ 38%] Built target core
Scanning dependencies of target platform
[ 38%] Building C object
DocFormats/platform/CMakeFiles/platform.dir/src/Wrapper.c.o
[ 66%] Built target platform
[ 67%] Built target unittest
[ 96%] Built target ooxml
[ 97%] Built target latex
Linking C static library ../lib/libDocFormats.a
[ 97%] Built target DocFormats
Linking C executable ../../../bin/dftest
[ 97%] Built target dftest
Linking C executable ../../../bin/dfconvert
[ 98%] Built target dfconvert
Linking C executable ../../../bin/dfutil
[100%] Built target dfutil
--
Visit my Coding Diary: http://gabriela-gibson.blogspot.com/
Re: More on the xmalloc project
Posted by jan i <ja...@apache.org>.
On Thursday, February 26, 2015, Gabriela Gibson <ga...@gmail.com>
wrote:
> Hi Peter, (and everyone else!)
>
> I read your JIRA comment, and that's an interesting thing with the missing
> headerfiles. My build report[1] was OK so, maybe different compilers take
> headers in a different order?
>
> Regards the clang build, should/could we have this as a regular build
> option in the CMake file?
>
> Another thing to notice was that my test had quite a lot of malloc 0 calls,
> so whatever is run there, maybe it could do with a check, or perhaps
> xmalloc should check for if (!size) ... ?
A simple if sounds correct, but it is legal so it should just return.
Funny side effect on my little embedded Linux kernel, free(0) leads to
process termination.
>
> The html file I downloaded and tested to get all those zeros was here:
>
>
> http://stackoverflow.com/questions/8880603/loop-through-array-of-strings-in-bash-script
>
> cmd I used to test was:
>
> ./bin/dfutil -normalize
>
> ~/cor2/incubator-corinthia/loop-through-array-of-strings-in-bash-script.html
>
> Towards the end, there's a heap of 0 allocations.
>
> G
>
> Ps.: Currently trying to figure out the best way of testing the mallocs in
> the WrapperTest.c file. Will holler when/if I get stuck.
Should be quite easy, expand the array up at the top, and write the test
functions.
remark you can start dftest so it runs only platform test cases (faster).
rgds
jan i
>
> [1] My build report:
>
> make
> [ 5%] Built target odf
> [ 7%] Built target api
> [ 38%] Built target core
> Scanning dependencies of target platform
> [ 38%] Building C object
> DocFormats/platform/CMakeFiles/platform.dir/src/Wrapper.c.o
> [ 66%] Built target platform
> [ 67%] Built target unittest
> [ 96%] Built target ooxml
> [ 97%] Built target latex
> Linking C static library ../lib/libDocFormats.a
> [ 97%] Built target DocFormats
> Linking C executable ../../../bin/dftest
> [ 97%] Built target dftest
> Linking C executable ../../../bin/dfconvert
> [ 98%] Built target dfconvert
> Linking C executable ../../../bin/dfutil
> [100%] Built target dfutil
>
> --
> Visit my Coding Diary: http://gabriela-gibson.blogspot.com/
>
--
Sent from My iPad, sorry for any misspellings.
Re: More on the xmalloc project
Posted by jan i <ja...@apache.org>.
On Friday, February 27, 2015, Dennis E. Hamilton <de...@acm.org>
wrote:
> I don't understand the thinking here.
>
> Let's step back a little bit.
>
> The reason for our introduction of an xmalloc() is to prevent return of
> NULL until there is a better approach to dealing with resource exhaustion
> based on the context and what the application is able to provide other than
> abrupt termination.
that was one of the reasons
>
> If the input parameter is 0 and the malloc() behavior is to return either
> NULL or a non-NULL but not-usable address, it would seem appropriate to not
> allows such cases in *our* xmalloc(), especially because of the potential
> need to free something (although free tends to quietly figure out when it
> need not actually do anything).
actually you nail the problem, which many memory managers have. You call
malloc(0) twice and then free(0) once,.....,,which function forgot to call
free(). This is the reason many memory managers, ask the caller to supply
an id.
>
> In this case, for the cases that apply to us, xmalloc() should treat an
> input of 0 as if malloc() returned NULL, but without calling malloc(),
> since presumably our code should never do that. (Using a debug assert
> might be better because of that, although my own preference is to not have
> debug builds provide different behavior than production builds.)
>
> Additional question: Is xmalloc() already a well-defined provision of some
> platform libraries? Then perhaps it is necessary to adopt a name that will
> not be confused with one of those?
xfoo is a de facto c standard for replacing clib functions.
rgds
jan i
>
> - Dennis
>
> -----Original Message-----
> From: Edward Zimmermann [mailto:Edward.Zimmermann@cib.de <javascript:;>]
> Sent: Friday, February 27, 2015 03:53
> To: dev@corinthia.incubator.apache.org <javascript:;>
> Subject: AW: More on the xmalloc project
>
> Correct malloc(0) is defined in the specification. What is, however,
> implementation defined is what is returned: a null pointer or a pointer
> with which one should not try to access its objects (since it is allocated
> with 0 bytes it can only be assumed that it is only the empty set). The
> implementation decision of null versus pointer with 0 bytes allocated is
> driven by factors beyond the scope of our concerns--- but there are some
> very specific use cases where it could be useful (think realloc where it
> does not need to move the block). NOTE: malloc(0) when it does not return a
> NULL can consume memory and needs to be freed-- since free(NULL) is OK we
> can readily free any pointer result of a malloc without worries.
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Peter Kelly [mailto:pmkelly@apache.org <javascript:;>]
> > Gesendet: Freitag, 27. Februar 2015 11:11
> > An: dev@corinthia.incubator.apache.org <javascript:;>
> > Betreff: Re: More on the xmalloc project
> >
> > > On 27 Feb 2015, at 3:59 am, Gabriela Gibson
> > <gabriela.gibson@gmail.com <javascript:;>> wrote:
> > >
> > > Hi Peter, (and everyone else!)
> > >
> > > I read your JIRA comment, and that's an interesting thing with the
> > > missing headerfiles. My build report[1] was OK so, maybe different
> > > compilers take headers in a different order?
> > >
> > > Regards the clang build, should/could we have this as a regular build
> > > option in the CMake file?
> >
> > I don’t really have an opinion on this personally… it’s the default
> > compiler on OS X. Depends if people want to use clang on linux/windows
> > as well.
> >
> > I wouldn’t regard it as a priority right now.
> >
> > > Another thing to notice was that my test had quite a lot of malloc 0
> > > calls, so whatever is run there, maybe it could do with a check, or
> > > perhaps xmalloc should check for if (!size) … ?
> >
> > It’s valid to call malloc with a request of 0 bytes.
> >
> > —
> > Dr Peter M. Kelly
> > pmkelly@apache.org <javascript:;>
> >
> > PGP key: http://www.kellypmk.net/pgp-key <http://www.kellypmk.net/pgp-
> > key> (fingerprint 5435 6718 59F0 DD1F BFA0 5E46 2523 BAA1 44AE 2966)
>
>
>
--
Sent from My iPad, sorry for any misspellings.
RE: More on the xmalloc project
Posted by "Dennis E. Hamilton" <de...@acm.org>.
I don't understand the thinking here.
Let's step back a little bit.
The reason for our introduction of an xmalloc() is to prevent return of NULL until there is a better approach to dealing with resource exhaustion based on the context and what the application is able to provide other than abrupt termination.
If the input parameter is 0 and the malloc() behavior is to return either NULL or a non-NULL but not-usable address, it would seem appropriate to not allows such cases in *our* xmalloc(), especially because of the potential need to free something (although free tends to quietly figure out when it need not actually do anything).
In this case, for the cases that apply to us, xmalloc() should treat an input of 0 as if malloc() returned NULL, but without calling malloc(), since presumably our code should never do that. (Using a debug assert might be better because of that, although my own preference is to not have debug builds provide different behavior than production builds.)
Additional question: Is xmalloc() already a well-defined provision of some platform libraries? Then perhaps it is necessary to adopt a name that will not be confused with one of those?
- Dennis
-----Original Message-----
From: Edward Zimmermann [mailto:Edward.Zimmermann@cib.de]
Sent: Friday, February 27, 2015 03:53
To: dev@corinthia.incubator.apache.org
Subject: AW: More on the xmalloc project
Correct malloc(0) is defined in the specification. What is, however, implementation defined is what is returned: a null pointer or a pointer with which one should not try to access its objects (since it is allocated with 0 bytes it can only be assumed that it is only the empty set). The implementation decision of null versus pointer with 0 bytes allocated is driven by factors beyond the scope of our concerns--- but there are some very specific use cases where it could be useful (think realloc where it does not need to move the block). NOTE: malloc(0) when it does not return a NULL can consume memory and needs to be freed-- since free(NULL) is OK we can readily free any pointer result of a malloc without worries.
> -----Ursprüngliche Nachricht-----
> Von: Peter Kelly [mailto:pmkelly@apache.org]
> Gesendet: Freitag, 27. Februar 2015 11:11
> An: dev@corinthia.incubator.apache.org
> Betreff: Re: More on the xmalloc project
>
> > On 27 Feb 2015, at 3:59 am, Gabriela Gibson
> <ga...@gmail.com> wrote:
> >
> > Hi Peter, (and everyone else!)
> >
> > I read your JIRA comment, and that's an interesting thing with the
> > missing headerfiles. My build report[1] was OK so, maybe different
> > compilers take headers in a different order?
> >
> > Regards the clang build, should/could we have this as a regular build
> > option in the CMake file?
>
> I don’t really have an opinion on this personally… it’s the default
> compiler on OS X. Depends if people want to use clang on linux/windows
> as well.
>
> I wouldn’t regard it as a priority right now.
>
> > Another thing to notice was that my test had quite a lot of malloc 0
> > calls, so whatever is run there, maybe it could do with a check, or
> > perhaps xmalloc should check for if (!size) … ?
>
> It’s valid to call malloc with a request of 0 bytes.
>
> —
> Dr Peter M. Kelly
> pmkelly@apache.org
>
> PGP key: http://www.kellypmk.net/pgp-key <http://www.kellypmk.net/pgp-
> key> (fingerprint 5435 6718 59F0 DD1F BFA0 5E46 2523 BAA1 44AE 2966)
AW: More on the xmalloc project
Posted by Edward Zimmermann <Ed...@cib.de>.
Correct malloc(0) is defined in the specification. What is, however, implementation defined is what is returned: a null pointer or a pointer with which one should not try to access its objects (since it is allocated with 0 bytes it can only be assumed that it is only the empty set). The implementation decision of null versus pointer with 0 bytes allocated is driven by factors beyond the scope of our concerns--- but there are some very specific use cases where it could be useful (think realloc where it does not need to move the block). NOTE: malloc(0) when it does not return a NULL can consume memory and needs to be freed-- since free(NULL) is OK we can readily free any pointer result of a malloc without worries.
> -----Ursprüngliche Nachricht-----
> Von: Peter Kelly [mailto:pmkelly@apache.org]
> Gesendet: Freitag, 27. Februar 2015 11:11
> An: dev@corinthia.incubator.apache.org
> Betreff: Re: More on the xmalloc project
>
> > On 27 Feb 2015, at 3:59 am, Gabriela Gibson
> <ga...@gmail.com> wrote:
> >
> > Hi Peter, (and everyone else!)
> >
> > I read your JIRA comment, and that's an interesting thing with the
> > missing headerfiles. My build report[1] was OK so, maybe different
> > compilers take headers in a different order?
> >
> > Regards the clang build, should/could we have this as a regular build
> > option in the CMake file?
>
> I don’t really have an opinion on this personally… it’s the default
> compiler on OS X. Depends if people want to use clang on linux/windows
> as well.
>
> I wouldn’t regard it as a priority right now.
>
> > Another thing to notice was that my test had quite a lot of malloc 0
> > calls, so whatever is run there, maybe it could do with a check, or
> > perhaps xmalloc should check for if (!size) … ?
>
> It’s valid to call malloc with a request of 0 bytes.
>
> —
> Dr Peter M. Kelly
> pmkelly@apache.org
>
> PGP key: http://www.kellypmk.net/pgp-key <http://www.kellypmk.net/pgp-
> key> (fingerprint 5435 6718 59F0 DD1F BFA0 5E46 2523 BAA1 44AE 2966)
Re: More on the xmalloc project
Posted by Peter Kelly <pm...@apache.org>.
> On 27 Feb 2015, at 3:59 am, Gabriela Gibson <ga...@gmail.com> wrote:
>
> Hi Peter, (and everyone else!)
>
> I read your JIRA comment, and that's an interesting thing with the missing
> headerfiles. My build report[1] was OK so, maybe different compilers take
> headers in a different order?
>
> Regards the clang build, should/could we have this as a regular build
> option in the CMake file?
I don’t really have an opinion on this personally… it’s the default compiler on OS X. Depends if people want to use clang on linux/windows as well.
I wouldn’t regard it as a priority right now.
> Another thing to notice was that my test had quite a lot of malloc 0 calls,
> so whatever is run there, maybe it could do with a check, or perhaps
> xmalloc should check for if (!size) … ?
It’s valid to call malloc with a request of 0 bytes.
—
Dr Peter M. Kelly
pmkelly@apache.org
PGP key: http://www.kellypmk.net/pgp-key <http://www.kellypmk.net/pgp-key>
(fingerprint 5435 6718 59F0 DD1F BFA0 5E46 2523 BAA1 44AE 2966)