You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by William A Rowe Jr <wr...@rowe-clan.net> on 2015/11/20 19:31:57 UTC

Optimization, modern C and APR 2.0 onwards

I'm wondering how the group would react to refactoring some of APR 2.0
to either offer inline code for many of our heavily consumed functions,
or offering inline + fn implementations alongside one another?

Would it still be necessary in this day and age to support C compilers
that do not support inline at all, e.g. hide the inline declarations based
on some macro switch leaving only the function stub?

We can obviously debate the merits of which functions are most
prime for optimization, including how mature each is (due to the
fact that the user will be 'stuck' with the implementation until they
recompile their own code against a new release of apr in the event
of a bug or security fix).

Thoughts?

Bill

Re: Optimization, modern C and APR 2.0 onwards

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Nov 23, 2015 at 1:42 PM, Ruediger Pluem <rp...@apache.org> wrote:

>
> On 11/20/2015 07:31 PM, William A Rowe Jr wrote:
> > I'm wondering how the group would react to refactoring some of APR 2.0
> > to either offer inline code for many of our heavily consumed functions,
> > or offering inline + fn implementations alongside one another?
> >
> > Would it still be necessary in this day and age to support C compilers
> > that do not support inline at all, e.g. hide the inline declarations
> based
> > on some macro switch leaving only the function stub?
> >
> > We can obviously debate the merits of which functions are most
> > prime for optimization, including how mature each is (due to the
> > fact that the user will be 'stuck' with the implementation until they
> > recompile their own code against a new release of apr in the event
> > of a bug or security fix).
>
> Especially in case of security fixes it would be a real pain to require
> all consumers of the library to recompile. This is one of the beauties of
> dynamic linking :-).
> So I would like to see it limited to very stable functions that exist for
> a long time unmodified
> and that have a small code base or we should give the developers a choice
> by offering
> both ways (inline + fn).
>

I think that's inevitable (offering the choice). Florian reminded me that
compiler support of inline as well as compiler support for partially
declared structures will probably be necessary to optimize the code.
So we have at least two feature tests (and a combined resulting
APR_USE_INLINED macro) that controls whether this is available,
and one that should be easily disabled on a source-by-source basis.

We will still have to leave more complex code in the library, not
exposed to in-lining. But I expect the most significant optimization
to come from some of the simplest functions, keeping the complexity
of APR mostly opaque to the consumer's sources.

Re: Optimization, modern C and APR 2.0 onwards

Posted by Ruediger Pluem <rp...@apache.org>.

On 11/20/2015 07:31 PM, William A Rowe Jr wrote:
> I'm wondering how the group would react to refactoring some of APR 2.0
> to either offer inline code for many of our heavily consumed functions,
> or offering inline + fn implementations alongside one another?
> 
> Would it still be necessary in this day and age to support C compilers
> that do not support inline at all, e.g. hide the inline declarations based
> on some macro switch leaving only the function stub?
> 
> We can obviously debate the merits of which functions are most
> prime for optimization, including how mature each is (due to the
> fact that the user will be 'stuck' with the implementation until they
> recompile their own code against a new release of apr in the event
> of a bug or security fix).

Especially in case of security fixes it would be a real pain to require
all consumers of the library to recompile. This is one of the beauties of dynamic linking :-).
So I would like to see it limited to very stable functions that exist for a long time unmodified
and that have a small code base or we should give the developers a choice by offering
both ways (inline + fn).

Regards

Rüdiger

Re: Optimization, modern C and APR 2.0 onwards

Posted by Florian Weimer <fw...@deneb.enyo.de>.
* William A. Rowe, Jr.:

> I'm wondering how the group would react to refactoring some of APR 2.0
> to either offer inline code for many of our heavily consumed functions,
> or offering inline + fn implementations alongside one another?

The downside of inline functions is a much larger ABI.  Inline
functions can only access non-opaque structs, and client code will
hard-code field offset (and maybe struct sizes, too).  Even minimal
changes to struct definitions or inline function definitions will
require recompilation of all client code using it (at least from a
formal point of view).

Florian

Re: Optimization, modern C and APR 2.0 onwards

Posted by Jim Jagielski <ji...@jaguNET.com>.
Yeah, it was the hash improvement (in fact, I use that exact
algo w/ proxy) as well as the pool enhancements (esp related
to fragments) which would be super cool to pull in.


> On Nov 20, 2015, at 6:08 PM, Greg Stein <gs...@gmail.com> wrote:
> 
> On Fri, Nov 20, 2015 at 4:19 PM, Yann Ylavic <yl...@gmail.com> wrote:
> On Fri, Nov 20, 2015 at 11:14 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> > On Fri, Nov 20, 2015 at 2:55 PM, Yann Ylavic <yl...@gmail.com> wrote:
> >>
> >> On Fri, Nov 20, 2015 at 8:00 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> >> > If we are serious about having a serious update to APR, I
> >> > would recommend that we use more up-to-date data structures,
> >> > patterns and algorithms than those in apr1. For example,
> >> > Greg's pocore mini lib is an example of the types of improvements
> >> > we should consider.
> >>
> >> Could you share a pointer to it please?
> >
> >
> > https://github.com/gstein/pocore
> 
> Thanks!
> 
> Jim may have other items in mind, but I believe its notable improvements are:
> 
> * hash tables built upon modern research
> * pools that avoid long-term memory fragmentation, including free() capability
> * structured error handling
> 
> As a solid test of the pocore library, I patched APR to use pocore's pools and hash tables (see apr/branches/gstein-pocore/) and then ran Subversion's extensive test suite. Except for tests depending upon hash table iteration, I got it all working just fine. pocore's pools are faster than APR (tho both are screaming fast, in the scheme of things), APR pools on top of pocore kinda slowed down the tests :-P
> 
> I'd be more than happy to discuss its design, both now and future direction (eg. memory coalescing is intended, but not yet implemented).
> 
> Cheers,
> -g
>

Re: Optimization, modern C and APR 2.0 onwards

Posted by Greg Stein <gs...@gmail.com>.
On Fri, Nov 20, 2015 at 4:19 PM, Yann Ylavic <yl...@gmail.com> wrote:

> On Fri, Nov 20, 2015 at 11:14 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> > On Fri, Nov 20, 2015 at 2:55 PM, Yann Ylavic <yl...@gmail.com>
> wrote:
> >>
> >> On Fri, Nov 20, 2015 at 8:00 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> >> > If we are serious about having a serious update to APR, I
> >> > would recommend that we use more up-to-date data structures,
> >> > patterns and algorithms than those in apr1. For example,
> >> > Greg's pocore mini lib is an example of the types of improvements
> >> > we should consider.
> >>
> >> Could you share a pointer to it please?
> >
> >
> > https://github.com/gstein/pocore
>
> Thanks!
>

Jim may have other items in mind, but I believe its notable improvements
are:

* hash tables built upon modern research
* pools that avoid long-term memory fragmentation, including free()
capability
* structured error handling

As a solid test of the pocore library, I patched APR to use pocore's pools
and hash tables (see apr/branches/gstein-pocore/) and then ran Subversion's
extensive test suite. Except for tests depending upon hash table iteration,
I got it all working just fine. pocore's pools are faster than APR (tho
both are screaming fast, in the scheme of things), APR pools on top of
pocore kinda slowed down the tests :-P

I'd be more than happy to discuss its design, both now and future direction
(eg. memory coalescing is intended, but not yet implemented).

Cheers,
-g

Re: Optimization, modern C and APR 2.0 onwards

Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Nov 20, 2015 at 11:14 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Fri, Nov 20, 2015 at 2:55 PM, Yann Ylavic <yl...@gmail.com> wrote:
>>
>> On Fri, Nov 20, 2015 at 8:00 PM, Jim Jagielski <ji...@jagunet.com> wrote:
>> > If we are serious about having a serious update to APR, I
>> > would recommend that we use more up-to-date data structures,
>> > patterns and algorithms than those in apr1. For example,
>> > Greg's pocore mini lib is an example of the types of improvements
>> > we should consider.
>>
>> Could you share a pointer to it please?
>
>
> https://github.com/gstein/pocore

Thanks!

Re: Optimization, modern C and APR 2.0 onwards

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Fri, Nov 20, 2015 at 2:55 PM, Yann Ylavic <yl...@gmail.com> wrote:

> On Fri, Nov 20, 2015 at 8:00 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> > If we are serious about having a serious update to APR, I
> > would recommend that we use more up-to-date data structures,
> > patterns and algorithms than those in apr1. For example,
> > Greg's pocore mini lib is an example of the types of improvements
> > we should consider.
>
> Could you share a pointer to it please?
>

https://github.com/gstein/pocore

Re: Optimization, modern C and APR 2.0 onwards

Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Nov 20, 2015 at 8:00 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> If we are serious about having a serious update to APR, I
> would recommend that we use more up-to-date data structures,
> patterns and algorithms than those in apr1. For example,
> Greg's pocore mini lib is an example of the types of improvements
> we should consider.

Could you share a pointer to it please?

Thanks,
Yann.

Re: Optimization, modern C and APR 2.0 onwards

Posted by Jim Jagielski <ji...@jaguNET.com>.
If we are serious about having a serious update to APR, I
would recommend that we use more up-to-date data structures,
patterns and algorithms than those in apr1. For example,
Greg's pocore mini lib is an example of the types of improvements
we should consider.

> On Nov 20, 2015, at 1:31 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> 
> I'm wondering how the group would react to refactoring some of APR 2.0
> to either offer inline code for many of our heavily consumed functions,
> or offering inline + fn implementations alongside one another?
> 
> Would it still be necessary in this day and age to support C compilers
> that do not support inline at all, e.g. hide the inline declarations based
> on some macro switch leaving only the function stub?
> 
> We can obviously debate the merits of which functions are most
> prime for optimization, including how mature each is (due to the
> fact that the user will be 'stuck' with the implementation until they
> recompile their own code against a new release of apr in the event
> of a bug or security fix).
> 
> Thoughts?
> 
> Bill

Re: Optimization, modern C and APR 2.0 onwards

Posted by Branko Čibej <br...@apache.org>.
On 20.11.2015 19:53, Bert Huijben wrote:
> +1
>
>  
>
> As long as we don’t require complete/100% C99 at this time.
>
>  
>
> Microsoft only intends to implement the C99 subset that is also part of the recent C++ specs (or just easy to do) in Visual Studio, and in most cases it already does in the most recent version.
>
>  
>
> But talking specifically about inline… I think it had that in VC 6.0/1998.

Microsoft C/C++ 6.0 as a matter of fact ... on DOS, 10 years earlier, a
couple years before Visual C++ 1.0. :)

> Personally my interest stops below the VS 2008 version. I won’t object on breaking support for older versions.
>
> (This is +- what we support with Subversion. VS2005 should still work as there are not many differences with 2008, but nobody in the project tested anything older in a long time.)
>
>  
>
>                 Bert
>
>  
>
> From: William A Rowe Jr [mailto:wrowe@rowe-clan.net] 
> Sent: vrijdag 20 november 2015 19:32
> To: APR Developer List <de...@apr.apache.org>
> Subject: Optimization, modern C and APR 2.0 onwards
>
>  
>
> I'm wondering how the group would react to refactoring some of APR 2.0
>
> to either offer inline code for many of our heavily consumed functions,
>
> or offering inline + fn implementations alongside one another?
>
>  
>
> Would it still be necessary in this day and age to support C compilers
>
> that do not support inline at all, e.g. hide the inline declarations based
>
> on some macro switch leaving only the function stub?
>
>  
>
> We can obviously debate the merits of which functions are most
>
> prime for optimization, including how mature each is (due to the
>
> fact that the user will be 'stuck' with the implementation until they
>
> recompile their own code against a new release of apr in the event
>
> of a bug or security fix).
>
>  
>
> Thoughts?
>
>  
>
> Bill
>
>

RE: Optimization, modern C and APR 2.0 onwards

Posted by Bert Huijben <be...@qqmail.nl>.
+1

 

As long as we don’t require complete/100% C99 at this time.

 

Microsoft only intends to implement the C99 subset that is also part of the recent C++ specs (or just easy to do) in Visual Studio, and in most cases it already does in the most recent version.

 

But talking specifically about inline… I think it had that in VC 6.0/1998. 

 

Personally my interest stops below the VS 2008 version. I won’t object on breaking support for older versions.

(This is +- what we support with Subversion. VS2005 should still work as there are not many differences with 2008, but nobody in the project tested anything older in a long time.)

 

                Bert

 

From: William A Rowe Jr [mailto:wrowe@rowe-clan.net] 
Sent: vrijdag 20 november 2015 19:32
To: APR Developer List <de...@apr.apache.org>
Subject: Optimization, modern C and APR 2.0 onwards

 

I'm wondering how the group would react to refactoring some of APR 2.0

to either offer inline code for many of our heavily consumed functions,

or offering inline + fn implementations alongside one another?

 

Would it still be necessary in this day and age to support C compilers

that do not support inline at all, e.g. hide the inline declarations based

on some macro switch leaving only the function stub?

 

We can obviously debate the merits of which functions are most

prime for optimization, including how mature each is (due to the

fact that the user will be 'stuck' with the implementation until they

recompile their own code against a new release of apr in the event

of a bug or security fix).

 

Thoughts?

 

Bill