You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Jon Travis <jt...@covalent.net> on 2001/10/01 22:43:55 UTC
apr_vformatter changes & optimizations
I was thinking today that it would be nice to be able to pre-parse
*rintf style format strings into an intermediate array. This would
have the potential to speed up code which uses *rintf by quite a bit.
The only drawbacks that I can think of, is that the parser function
will need to take in a pool, in order to return the array, and that
means functions like snprintf(), etc. (which will use the preparser)
will need to use pools. I guess apr_snprintf, could just create its
own temporary pool, and destroy it before it returns.
Any thoughts on this?
-- Jon
Re: apr_vformatter changes & optimizations
Posted by Jon Travis <jt...@covalent.net>.
On Tue, Oct 02, 2001 at 09:14:51AM -0700, Brian Pane wrote:
> Greg Stein wrote:
>
> >On Mon, Oct 01, 2001 at 08:28:17PM -0700, Jon Travis wrote:
> >
> >>On Mon, Oct 01, 2001 at 04:07:13PM -0700, Greg Stein wrote:
> >>
> [...]
>
> >>>Personally, I'm guessing that more time is spent assembling than parsing.
> >>>
> >>Since the parsing code is intermingled with the code that throws it into
> >>the final buffer, there isn't a good way of profiling this right now.
> >>In order to figure out if we are, or not, I'll have to actually write the
> >>code to just-parse or just-output the data, which is the end result of
> >>my question, anyway.
> >>
> >
> >Well, if you want to do the code, who's to stop you? :-) But I would think
> >it ought to be perf-tested before it goes in.
> >
>
> In the case of Apache, most of the time spent in *printf is due to calls
> in ap_make_etag() and log_request_time().
>
> apr_vformatter() represents about 0.25% of the total CPU time in the httpd,
> so it's probably worth optimizing (there aren't many >1% optimizations left,
> except for tables and dir-merge). But we might get the same improvement
> more easily by optimizing away the *printf calls in ap_make_etag() and
> log_request_time() instead of trying to optimize the formatter itself.
I've written up a preparser and stripped-down vformatter, but the performance
increase is small, and not really worth the split-up code. It is probably
not even worth it to post the source here. It does, however, give me the
functionality I need (ability to examine format strings for specific
info as well as concoct new va_list stylee format lists)
-- Jon
Re: apr_vformatter changes & optimizations
Posted by Brian Pane <bp...@pacbell.net>.
Greg Stein wrote:
>On Mon, Oct 01, 2001 at 08:28:17PM -0700, Jon Travis wrote:
>
>>On Mon, Oct 01, 2001 at 04:07:13PM -0700, Greg Stein wrote:
>>
[...]
>>>Personally, I'm guessing that more time is spent assembling than parsing.
>>>
>>Since the parsing code is intermingled with the code that throws it into
>>the final buffer, there isn't a good way of profiling this right now.
>>In order to figure out if we are, or not, I'll have to actually write the
>>code to just-parse or just-output the data, which is the end result of
>>my question, anyway.
>>
>
>Well, if you want to do the code, who's to stop you? :-) But I would think
>it ought to be perf-tested before it goes in.
>
In the case of Apache, most of the time spent in *printf is due to calls
in ap_make_etag() and log_request_time().
apr_vformatter() represents about 0.25% of the total CPU time in the httpd,
so it's probably worth optimizing (there aren't many >1% optimizations left,
except for tables and dir-merge). But we might get the same improvement
more easily by optimizing away the *printf calls in ap_make_etag() and
log_request_time() instead of trying to optimize the formatter itself.
--Brian
Re: apr_vformatter changes & optimizations
Posted by Jon Travis <jt...@covalent.net>.
On Mon, Oct 01, 2001 at 08:51:54PM -0700, Greg Stein wrote:
> On Mon, Oct 01, 2001 at 08:28:17PM -0700, Jon Travis wrote:
> > On Mon, Oct 01, 2001 at 04:07:13PM -0700, Greg Stein wrote:
> > > On Mon, Oct 01, 2001 at 01:43:55PM -0700, Jon Travis wrote:
> > > > I was thinking today that it would be nice to be able to pre-parse
> > > > *rintf style format strings into an intermediate array. This would
> > > > have the potential to speed up code which uses *rintf by quite a bit.
> > >
> > > Potential vs. demonstrated?
> > >
> > > Can you show that we're spendig significant time parsing the format strings?
> > > Going and throwing a bunch of complexity for a small speedup might not be
> > > worthwhile :-)
> > >
> > > Personally, I'm guessing that more time is spent assembling than parsing.
> >
> > Since the parsing code is intermingled with the code that throws it into
> > the final buffer, there isn't a good way of profiling this right now.
> > In order to figure out if we are, or not, I'll have to actually write the
> > code to just-parse or just-output the data, which is the end result of
> > my question, anyway.
>
> Well, if you want to do the code, who's to stop you? :-) But I would think
> it ought to be perf-tested before it goes in.
Agreed. Working on it now.
> > Another useful feature of being able to pre-parse the list would be to
> > have the ability to examine the datatypes required for the format. Why
> > is this terribly useful, you ask? Well, v*printf takes a va_list, which
> > cannot portably be constructed at run-time. However, if we had a special
> > apr_va_list, which we could take as an argument to our special formatter
> > (which also took a preparsed array), we could generate and use v*printf
> > formats/arguments at runtime.
>
> Assume that you create a sentinel-terminated list of non-opaque structures.
> An app could then programmatically declare that stuff ahead of time and skip
> the whole parse step.
>
> Look at it as a sequence of instructions for assembling bits into a string.
Sure, you could skip the whole parse phase and generate the list yourself.
That is my end-goal here. Nobody would ever want to get rid of the parsing
code, though.
Either way, it would be nice to be able to (at run time) construct va_list
style arguments to pass off to a v*printf function. That is a bit of a
poor point in C. :-(
-- Jon
Re: apr_vformatter changes & optimizations
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Oct 01, 2001 at 08:28:17PM -0700, Jon Travis wrote:
> On Mon, Oct 01, 2001 at 04:07:13PM -0700, Greg Stein wrote:
> > On Mon, Oct 01, 2001 at 01:43:55PM -0700, Jon Travis wrote:
> > > I was thinking today that it would be nice to be able to pre-parse
> > > *rintf style format strings into an intermediate array. This would
> > > have the potential to speed up code which uses *rintf by quite a bit.
> >
> > Potential vs. demonstrated?
> >
> > Can you show that we're spendig significant time parsing the format strings?
> > Going and throwing a bunch of complexity for a small speedup might not be
> > worthwhile :-)
> >
> > Personally, I'm guessing that more time is spent assembling than parsing.
>
> Since the parsing code is intermingled with the code that throws it into
> the final buffer, there isn't a good way of profiling this right now.
> In order to figure out if we are, or not, I'll have to actually write the
> code to just-parse or just-output the data, which is the end result of
> my question, anyway.
Well, if you want to do the code, who's to stop you? :-) But I would think
it ought to be perf-tested before it goes in.
> Another useful feature of being able to pre-parse the list would be to
> have the ability to examine the datatypes required for the format. Why
> is this terribly useful, you ask? Well, v*printf takes a va_list, which
> cannot portably be constructed at run-time. However, if we had a special
> apr_va_list, which we could take as an argument to our special formatter
> (which also took a preparsed array), we could generate and use v*printf
> formats/arguments at runtime.
Assume that you create a sentinel-terminated list of non-opaque structures.
An app could then programmatically declare that stuff ahead of time and skip
the whole parse step.
Look at it as a sequence of instructions for assembling bits into a string.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: apr_vformatter changes & optimizations
Posted by Jon Travis <jt...@covalent.net>.
On Mon, Oct 01, 2001 at 04:07:13PM -0700, Greg Stein wrote:
> On Mon, Oct 01, 2001 at 01:43:55PM -0700, Jon Travis wrote:
> > I was thinking today that it would be nice to be able to pre-parse
> > *rintf style format strings into an intermediate array. This would
> > have the potential to speed up code which uses *rintf by quite a bit.
>
> Potential vs. demonstrated?
>
> Can you show that we're spendig significant time parsing the format strings?
> Going and throwing a bunch of complexity for a small speedup might not be
> worthwhile :-)
>
> Personally, I'm guessing that more time is spent assembling than parsing.
Since the parsing code is intermingled with the code that throws it into
the final buffer, there isn't a good way of profiling this right now.
In order to figure out if we are, or not, I'll have to actually write the
code to just-parse or just-output the data, which is the end result of
my question, anyway.
Another useful feature of being able to pre-parse the list would be to
have the ability to examine the datatypes required for the format. Why
is this terribly useful, you ask? Well, v*printf takes a va_list, which
cannot portably be constructed at run-time. However, if we had a special
apr_va_list, which we could take as an argument to our special formatter
(which also took a preparsed array), we could generate and use v*printf
formats/arguments at runtime.
-- Jon
Re: apr_vformatter changes & optimizations
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Oct 01, 2001 at 01:43:55PM -0700, Jon Travis wrote:
> I was thinking today that it would be nice to be able to pre-parse
> *rintf style format strings into an intermediate array. This would
> have the potential to speed up code which uses *rintf by quite a bit.
Potential vs. demonstrated?
Can you show that we're spendig significant time parsing the format strings?
Going and throwing a bunch of complexity for a small speedup might not be
worthwhile :-)
Personally, I'm guessing that more time is spent assembling than parsing.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/