You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Wendell Hatcher <we...@gmail.com> on 2020/12/11 17:54:40 UTC
[users@httpd] APR util slotmem errors.
Quick question how does the apr use the shm segments and why does it have a
slotmem error if we use mod_proxy with several balancer name calls and
multiple hosts apache servers on a single dev box? I am really trying to
understand how this code segment below works?
shm.c file call?
#if APR_USE_SHMEM_SHMGET 71 static key_t our_ftok(const char
*filename) 72 { 73 /* to help avoid collisions while still
using 74 * an easily recreated proj_id */ 75 apr_ssize_t
slen = strlen(filename); 76 return ftok(filename, 77
(int)apr_hashfunc_default(filename, &slen)); 78 } 79 #endif
APR_PERMS_SET_IMPLEMENT(shm)
696 {
697 #if APR_USE_SHMEM_SHMGET || APR_USE_SHMEM_SHMGET_ANON
698 struct shmid_ds shmbuf;
699 int shmid;
700 apr_shm_t *m = (apr_shm_t *)theshm;
701
702 if ((shmid = shmget(m->shmkey, 0, SHM_R | SHM_W)) == -1) {
703 return errno;
704 }
705 shmbuf.shm_perm.uid = uid;
706 shmbuf.shm_perm.gid = gid;
707 shmbuf.shm_perm.mode = apr_unix_perms2mode(perms);
708 if (shmctl(shmid, IPC_SET, &shmbuf) == -1) {
709 return errno;
710 }
711 return APR_SUCCESS;
712 #else
713 return APR_ENOTIMPL;
714 #endif
715 }
716
shmbuf.shm_perm.uid = uid;
706 shmbuf.shm_perm.gid = gid;
707 shmbuf.shm_perm.mode = apr_unix_perms2mode(perms);
[users@httpd] Re: APR util slotmem errors.
Posted by Wendell Hatcher <we...@gmail.com>.
>
> Quick question how does the apr use the shm segments and why does it have
> a slotmem error if we use mod_proxy with several balancer name calls and
> multiple hosts apache servers on a single dev box? I am really trying to
> understand how this code segment below works?
>
> shm.c file call?
>
> #if APR_USE_SHMEM_SHMGET 71 static key_t our_ftok(const char *filename) 72 { 73 /* to help avoid collisions while still using 74 * an easily recreated proj_id */ 75 apr_ssize_t slen = strlen(filename); 76 return ftok(filename, 77 (int)apr_hashfunc_default(filename, &slen)); 78 } 79 #endif
>
>
>
>
> APR_PERMS_SET_IMPLEMENT(shm)
> 696 {
> 697 #if APR_USE_SHMEM_SHMGET || APR_USE_SHMEM_SHMGET_ANON
> 698 struct shmid_ds shmbuf;
> 699 int shmid;
> 700 apr_shm_t *m = (apr_shm_t *)theshm;
> 701
> 702 if ((shmid = shmget(m->shmkey, 0, SHM_R | SHM_W)) == -1) {
> 703 return errno;
> 704 }
> 705 shmbuf.shm_perm.uid = uid;
> 706 shmbuf.shm_perm.gid = gid;
> 707 shmbuf.shm_perm.mode = apr_unix_perms2mode(perms);
> 708 if (shmctl(shmid, IPC_SET, &shmbuf) == -1) {
> 709 return errno;
> 710 }
> 711 return APR_SUCCESS;
> 712 #else
> 713 return APR_ENOTIMPL;
> 714 #endif
> 715 }
> 716
> shmbuf.shm_perm.uid = uid;
> 706 shmbuf.shm_perm.gid = gid;
> 707 shmbuf.shm_perm.mode = apr_unix_perms2mode(perms);
>
Re: [users@httpd] APR util slotmem errors.
Posted by Wendell Hatcher <we...@gmail.com>.
Thanks Yann, I really appreciate the information! This really helps!
On Sat, Dec 12, 2020, 7:01 AM Yann Ylavic <yl...@gmail.com> wrote:
> Hi,
>
> These are more questions for the dev@apr.a.o (or dev@httpd) mailing
> list, though there are APR developers on this list too ;)
>
> >
> > Quick question how does the apr use the shm segments and why does it
> have a slotmem error if we use mod_proxy with several balancer name calls
> and multiple hosts apache servers on a single dev box? I am really trying
> to understand how this code segment below works?
>
> So you don't have balancer://url duplicates (anymore) and still slotmem
> errors?
>
> >
> > shm.c file call?
> >
> > #if APR_USE_SHMEM_SHMGET
> > 71 static key_t our_ftok(const char *filename)
> > 72 {
> > 73 /* to help avoid collisions while still using
> > 74 * an easily recreated proj_id */
> > 75 apr_ssize_t slen = strlen(filename);
> > 76 return ftok(filename,
> > 77 (int)apr_hashfunc_default(filename, &slen));
> > 78 }
> > 79 #endif
>
> This is a wrapper around the system's ftok() function, a thingy needed
> by the IPC SysV API to create a unique ID from a file path, to be
> passed to shmget() & co system calls.
>
> From the Linux man page:
>
> SYNOPSIS
> key_t ftok(const char *pathname, int proj_id);
>
> DESCRIPTION
> The ftok() function uses the identity of the file named by the
> given pathname (which must refer to an existing, accessible file) and
> the least significant 8 bits of proj_id (which must be non‐zero) to
> generate a key_t type System V IPC key, suitable for use with
> msgget(2), semget(2), or shmget(2).
> The resulting value is the same for all pathnames that name the
> same file, when the same value of proj_id is used.
> The value returned should be different when the (simultaneously
> existing) files or the project IDs differ.
>
> NOTES
> On some ancient systems, the prototype was:
> key_t ftok(char *pathname, char proj_id);
> Today, proj_id is an int, but still only 8 bits are used.
> Typical usage has an ASCII character proj_id, that is why the
> behavior is said to be undefined when proj_id is zero.
> Of course, no guarantee can be given that the resulting key_t is
> unique.
> Typically, a best-effort attempt combines the given proj_id
> byte, the lower 16 bits of the inode number, and the lower 8 bits of
> the device number into a 32-bit result.
> Collisions may easily happen, for example between files on
> /dev/hda1 and files on /dev/sda1.
>
> Neat.. the IPC SysV API is horrid (IMHO) :/
>
> Fortunately the APR lib does not expose this proj_id since it has no
> meaning for the other possible SHM mechanisms (e.g. POSIX).
> To help with the collision issue, the proj_id is not fixed to a
> non-zero constant either, but rather hashed from the filename to
> improve mixing.
>
> The apr_hashfunc_default() function used here (djbhash) is not the
> more collision resistant one.
> For the POSIX mechanism the APR lib also mixes in an rshash of the
> filename, for IPC SysV this would be:
>
> static key_t our_ftok(const char *filename)
> {
> /* to help avoid collisions while still using
> * an easily recreated proj_id */
> apr_ssize_t flen;
> unsigned int h;
>
> flen = strlen(filename);
> h = apr_hashfunc_default(filename, &flen);
> h ^= rshash(filename);
> if (h == 0) {
> h = 0xc; /* arbitrary, non-zero */
> }
> return ftok(filename, h);
> }
>
> But there have been no issue raised so far for the current IPC SysV
> implementation.
> Do you observe collisions for different file names here, by e.g.
> adding a printf of the filename and hash in the current our_ftok()
> function?
>
> >
> > APR_PERMS_SET_IMPLEMENT(shm)
> > 696 {
> > 697 #if APR_USE_SHMEM_SHMGET || APR_USE_SHMEM_SHMGET_ANON
> > 698 struct shmid_ds shmbuf;
> > 699 int shmid;
> > 700 apr_shm_t *m = (apr_shm_t *)theshm;
> > 701
> > 702 if ((shmid = shmget(m->shmkey, 0, SHM_R | SHM_W)) == -1) {
> > 703 return errno;
> > 704 }
>
> Here m->shmkey is then the result of our_ftok(filename).
>
>
> Regards;
> Yann.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
Re: [users@httpd] APR util slotmem errors.
Posted by Yann Ylavic <yl...@gmail.com>.
Hi,
These are more questions for the dev@apr.a.o (or dev@httpd) mailing
list, though there are APR developers on this list too ;)
>
> Quick question how does the apr use the shm segments and why does it have a slotmem error if we use mod_proxy with several balancer name calls and multiple hosts apache servers on a single dev box? I am really trying to understand how this code segment below works?
So you don't have balancer://url duplicates (anymore) and still slotmem errors?
>
> shm.c file call?
>
> #if APR_USE_SHMEM_SHMGET
> 71 static key_t our_ftok(const char *filename)
> 72 {
> 73 /* to help avoid collisions while still using
> 74 * an easily recreated proj_id */
> 75 apr_ssize_t slen = strlen(filename);
> 76 return ftok(filename,
> 77 (int)apr_hashfunc_default(filename, &slen));
> 78 }
> 79 #endif
This is a wrapper around the system's ftok() function, a thingy needed
by the IPC SysV API to create a unique ID from a file path, to be
passed to shmget() & co system calls.
From the Linux man page:
SYNOPSIS
key_t ftok(const char *pathname, int proj_id);
DESCRIPTION
The ftok() function uses the identity of the file named by the
given pathname (which must refer to an existing, accessible file) and
the least significant 8 bits of proj_id (which must be non‐zero) to
generate a key_t type System V IPC key, suitable for use with
msgget(2), semget(2), or shmget(2).
The resulting value is the same for all pathnames that name the
same file, when the same value of proj_id is used.
The value returned should be different when the (simultaneously
existing) files or the project IDs differ.
NOTES
On some ancient systems, the prototype was:
key_t ftok(char *pathname, char proj_id);
Today, proj_id is an int, but still only 8 bits are used.
Typical usage has an ASCII character proj_id, that is why the
behavior is said to be undefined when proj_id is zero.
Of course, no guarantee can be given that the resulting key_t is unique.
Typically, a best-effort attempt combines the given proj_id
byte, the lower 16 bits of the inode number, and the lower 8 bits of
the device number into a 32-bit result.
Collisions may easily happen, for example between files on
/dev/hda1 and files on /dev/sda1.
Neat.. the IPC SysV API is horrid (IMHO) :/
Fortunately the APR lib does not expose this proj_id since it has no
meaning for the other possible SHM mechanisms (e.g. POSIX).
To help with the collision issue, the proj_id is not fixed to a
non-zero constant either, but rather hashed from the filename to
improve mixing.
The apr_hashfunc_default() function used here (djbhash) is not the
more collision resistant one.
For the POSIX mechanism the APR lib also mixes in an rshash of the
filename, for IPC SysV this would be:
static key_t our_ftok(const char *filename)
{
/* to help avoid collisions while still using
* an easily recreated proj_id */
apr_ssize_t flen;
unsigned int h;
flen = strlen(filename);
h = apr_hashfunc_default(filename, &flen);
h ^= rshash(filename);
if (h == 0) {
h = 0xc; /* arbitrary, non-zero */
}
return ftok(filename, h);
}
But there have been no issue raised so far for the current IPC SysV
implementation.
Do you observe collisions for different file names here, by e.g.
adding a printf of the filename and hash in the current our_ftok()
function?
>
> APR_PERMS_SET_IMPLEMENT(shm)
> 696 {
> 697 #if APR_USE_SHMEM_SHMGET || APR_USE_SHMEM_SHMGET_ANON
> 698 struct shmid_ds shmbuf;
> 699 int shmid;
> 700 apr_shm_t *m = (apr_shm_t *)theshm;
> 701
> 702 if ((shmid = shmget(m->shmkey, 0, SHM_R | SHM_W)) == -1) {
> 703 return errno;
> 704 }
Here m->shmkey is then the result of our_ftok(filename).
Regards;
Yann.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org