You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2020/04/10 18:36:40 UTC
Memory management inside mod_proxy
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
First-time httpd hacker, here. I'm looking at:
https://bz.apache.org/bugzilla/show_bug.cgi?id=64338
I've found that around line 637 of mod_proxy_balancer.c it looks like
I just need to poke a value into the r->subprocess_env like this:
apt_table_setn(r->subprocess_env,
"BALANCER_WORKER_STATUS", XXX);
The question is what "XXX" should be. I need to convert
(*worker)->s->status (an unsigned int) into a char*.
I can manually malloc a char*, stick it in there, and deallocate it
each time back through, but that will leak memory when the pool is
recycled and I don't get a chance to free my char*.
I can use apt_table_set instead of apt_table_setn but that will make
performance suck even more than a single allocation each time
mod_proxy handles a request.
I can also use something like apr_itoa/apr_ltoa which will allocate
from the pool which presumably will be cleaned-up at some point,
but... when? I don't know anything about how APR/httpd handle memory
management and I'm afraid I'll much it all up if I don't ask this
question.
So ... what's the right way to do this?
A performance optimization is likely to be that most workers (a) have
all the same "everything is fine" status and (b) the status doesn't
change very often. So maybe having a table of predefined status
strings sitting around wouldn't be the worth possible thing in the
world; that would avoid many needless memory (re)allocations.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6QvLgACgkQHPApP6U8
pFgbURAAxphLP8xQwXS7cwBS5ptbTNp2tjAN55ASQjS6f0Z3novjtm1Q7NmWJKIc
G6W7ZV/tZTkMuU2xF1uTDP32RPCXZ+gi0SXdbnZdm1ZnpYZrGApbiTM+ymIBiUSU
3Sy/X5QavENDuHgaG+f+9Jq6a4BYYzPgvXmEt2lxgqrfCE2HqmbO4LRnYbeNNBDM
xu0esh9ZWKzMhfinYl/AIzWjMcEEEAU2gi5xOuq9luOg+LvQ8BnDchGoS9lCO4eO
vOuB6PesUEQ1fLxjCN/OUrNnC1y0OUua/Xfj7v/adZc8QoUdBoxz44x+BKl6reP6
cpqTPEmdPurwAQdFyvnMLkM5Simw+mkybCF184u0YseN+sOaPTrqP5WJufKn9lCF
q3CPDt3TuF0RucoMEGaY3W2Rw4NOc5yecFOG1vaVkIatrD81uMDZJCYo2wJpZiiK
esWSJ983frbfuerPRGJYKqUol4sPs2Zq5IzDUfWUBeOR+jYcqpFYoW0TKFpwDnAS
7+9dLzQjrW5RJpAdlv+0hkE3xGOACJazXgHr+Un6JJKaNgpp8ThaUVpkzN0gfX+d
TJwVGTxA2Vk5GFdSMeEof3v0vXEqQ4P6Wtfc/+glueUI5UxAs7wOukQd06cBgRo7
83gj/nnpxAq1EaRvWUwI1fY8mXflI0+/0sYclf+1Gmn/SKHB2GA=
=9SrP
-----END PGP SIGNATURE-----
Re: Memory management inside mod_proxy
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Eric,
On 4/10/20 15:56, Eric Covener wrote:
>>> Note: If you move to r->notes (also a table) you can use
>>> apr_table_setn to pass a casted pointer and retrieve the
>>> pointer on the other side. But this is probably a last
>>> resort.
>>
>> I'm not sure I follow, other than mechanically. Why might I want
>> to do that, exactly?
>
> Quick and dirty way for two modules to pass a reference to a
> struct around. mod_session providers use it almost like a cache.
Aha, okay. In this case, we're just setting an environment variable
and making sure we clean-up after ourselves. Any other modules who
want that data can find it in the environment :)
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6UhdwACgkQHPApP6U8
pFjTPA/9FTTZLBVzCS0n/5NXLwCWCZGYZdRC00ODh/Xu+Lo0OBPkAQ5vSH4zQigx
MS7Y3p7VCWEkQ9uxl+JDmxgXROsYbxbgbvl4xu1tK4kkLcoE7lBAye/rwJ3NA9YO
C1Xu6sO90cURtIAXgaD4pP7jT914H8LAhJ765U25VmuEkXjyhhzkGNO1PPENFWRL
44ijSB1d5HgzX5PJmll8AuawHHvKZnhWodiWTeOl6jgZxmP6caGp0SlgNfZoSddx
L3H4S+B5pyJCP2EkZ7EtXCTMVDqrHqsRlazFLji2GKHImO1t5BKh1psWQd64vwHi
wA8cumpaSk0YtXdl+uVAh3RK9v9Yk621PhlIDUjU7XhF4v8t34odCaw8K96yoWlN
TQULdCcIok869rEtONnfbB13pGTtG450JkWZPFatoU9Kkq5k9l4LAwAP+LCLYnK9
rL8IOw2arU1fZ9F2R3/60qZsPwD0UyuCfvUFgRv24BA64bg6Dek/z6XT1B92JI2x
NIPHOQvSA8bQMdcZuzfd0lNH8ofUSP8pVnaPWANFzwjhcUbmHADb0XkOihxebUIj
Gvx6npacu+H+V3uvHRZzcFKXlBtiqtzNcBlIMS4V/lx+1yXqFsP6wgzlCWshdPHW
R+5Qzl6ljlBzYfnd9VF7gjVnE81Az4t2usyTnFKyjhgYRuZPfT4=
=jJCz
-----END PGP SIGNATURE-----
Re: Memory management inside mod_proxy
Posted by Eric Covener <co...@gmail.com>.
> > Note: If you move to r->notes (also a table) you can use
> > apr_table_setn to pass a casted pointer and retrieve the pointer
> > on the other side. But this is probably a last resort.
>
> I'm not sure I follow, other than mechanically. Why might I want to do
> that, exactly?
Quick and dirty way for two modules to pass a reference to a struct
around. mod_session providers use it almost like a cache.
--
Eric Covener
covener@gmail.com
Re: Memory management inside mod_proxy
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Eric,
On 4/10/20 14:45, Eric Covener wrote:
> On Fri, Apr 10, 2020 at 2:36 PM Christopher Schultz
> <ch...@christopherschultz.net> wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>
>> All,
>>
>> First-time httpd hacker, here. I'm looking at:
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=64338
>>
>> I've found that around line 637 of mod_proxy_balancer.c it looks
>> like I just need to poke a value into the r->subprocess_env like
>> this:
>>
>> apt_table_setn(r->subprocess_env, "BALANCER_WORKER_STATUS",
>> XXX);
>>
>> The question is what "XXX" should be. I need to convert
>> (*worker)->s->status (an unsigned int) into a char*.
>>
>> I can manually malloc a char*, stick it in there, and deallocate
>> it each time back through, but that will leak memory when the
>> pool is recycled and I don't get a chance to free my char*.
>>
>> I can use apt_table_set instead of apt_table_setn but that will
>> make performance suck even more than a single allocation each
>> time mod_proxy handles a request.
>>
>> I can also use something like apr_itoa/apr_ltoa which will
>> allocate from the pool which presumably will be cleaned-up at
>> some point, but... when? I don't know anything about how
>> APR/httpd handle memory management and I'm afraid I'll much it
>> all up if I don't ask this question.
>>
>> So ... what's the right way to do this?
>>
>> A performance optimization is likely to be that most workers (a)
>> have all the same "everything is fine" status and (b) the status
>> doesn't change very often. So maybe having a table of predefined
>> status strings sitting around wouldn't be the worth possible
>> thing in the world; that would avoid many needless memory
>> (re)allocations.
>
> Hi Chris. You have access to a request_rec then you can allocate
> from r->pool. Allocations from there are automatically returned at
> the end of the request.
Ooh, okay. At least that's (a) safe and (b) more APR-ish than using
malloc/free.
> You could also just test the value in that status field and "setn"
> the variable to a constant character or string.
Right; there are no existing places (that I can find) where such a set
of constants already lives. I can add one if it makes sense to do so.
There are only so many combinations of statuses.
> Note: If you move to r->notes (also a table) you can use
> apr_table_setn to pass a casted pointer and retrieve the pointer
> on the other side. But this is probably a last resort.
I'm not sure I follow, other than mechanically. Why might I want to do
that, exactly?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6QygEACgkQHPApP6U8
pFh8JBAAyIhR4TJ0khZ2agTC1CaYlYxwB1yvxyNrg4x8Lgd0meR75jf8jbcw9/uC
3wVDuF678fe0fSkbFst2NPpp/BCb15NYCwqUSPaK93njI3jO/dANEdeHXl4yp32n
wE+y3RwMZ4BRiqZ0VqGM2bRI9wkMMgQSkeTyckW94BbUNKZpg2yYTD/PnPkj9a8I
irgPZ+kyv9b0M0EHUtBb5I9Mzv1kuU5dGnMvs3b4IaKiUf5PrHTAoOnc/Lc3NFr4
4Bdq6Z5wyciBxEnWMN+CGD5GUCp6MX6LT9jpTn8kIyDjdwxesixv87S2jvVZwK41
KDKQcUKLaQLUQ+3lCJmZefb0IG1cB07D6H/9tO9o89MyEVpih6743S743YtfLgLp
09ea46wzr5lOkkS8aMje8v+3uMDbUT9AX8od65N2MwlSEvyhKs0BOE8VK0Z4jD0L
+7syigere0xOYLs1aTDznocp6l6mb3A7GK+er8WRfL8CKAoX+smUk2dm/HkIG4Q5
h0xT4YFB2iEHT7GvdNoxyHpjeXCCVpfLTmy/UZvuBFtGn8ILAXEuksxw1Da/grNx
9SVj8Wu7Sd0oXjoeXila3N55T02iFIW4MWQlRAWdRc2Tr5+wu9y5Y5iLwPZ1Npef
YVXGSCurSrv2rVB61zzWEcKc0GZILKVnArKxbUC+65HdTfUB5CY=
=FY7e
-----END PGP SIGNATURE-----
Re: Memory management inside mod_proxy
Posted by Eric Covener <co...@gmail.com>.
On Fri, Apr 10, 2020 at 2:36 PM Christopher Schultz
<ch...@christopherschultz.net> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> First-time httpd hacker, here. I'm looking at:
> https://bz.apache.org/bugzilla/show_bug.cgi?id=64338
>
> I've found that around line 637 of mod_proxy_balancer.c it looks like
> I just need to poke a value into the r->subprocess_env like this:
>
> apt_table_setn(r->subprocess_env,
> "BALANCER_WORKER_STATUS", XXX);
>
> The question is what "XXX" should be. I need to convert
> (*worker)->s->status (an unsigned int) into a char*.
>
> I can manually malloc a char*, stick it in there, and deallocate it
> each time back through, but that will leak memory when the pool is
> recycled and I don't get a chance to free my char*.
>
> I can use apt_table_set instead of apt_table_setn but that will make
> performance suck even more than a single allocation each time
> mod_proxy handles a request.
>
> I can also use something like apr_itoa/apr_ltoa which will allocate
> from the pool which presumably will be cleaned-up at some point,
> but... when? I don't know anything about how APR/httpd handle memory
> management and I'm afraid I'll much it all up if I don't ask this
> question.
>
> So ... what's the right way to do this?
>
> A performance optimization is likely to be that most workers (a) have
> all the same "everything is fine" status and (b) the status doesn't
> change very often. So maybe having a table of predefined status
> strings sitting around wouldn't be the worth possible thing in the
> world; that would avoid many needless memory (re)allocations.
Hi Chris. You have access to a request_rec then you can allocate from r->pool.
Allocations from there are automatically returned at the end of the request.
You could also just test the value in that status field and "setn" the
variable to a constant character or string.
Note: If you move to r->notes (also a table) you can use
apr_table_setn to pass a casted pointer and retrieve the pointer on
the other side. But this is probably a last resort.