You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Harrie Hazewinkel <ha...@jrc.it> on 1998/01/20 14:35:14 UTC
Upgrade of the SNMP module for Apache 1.2.5
Hi,
I have upgraded the snmp module for the Apache 1.2.5 version.
I also fixed some changes as done within the MIB-definition
by the SYSAPPL-WG of the IETF and reduced memory-use for
the SNMP-engine. Therefore, I have created a tarball which
includes the 'patch' required in the original Apache code
(version 1.2.5) and the 'module/snmp' directory.
Info is available at
http://www-musiq.jrc.it/~harrie/software/
and the tarbal is at
http://www-musiq.jrc.it/~harrie/software/mod_snmp1b2_apache1.2.5.tar.gz
It is tested on Solaris2.5.1 and FreeBSD2.
So far I only encountered only a problem during a 'KILL -HUP'.
The SNMP-process is then not killed and it keeps the ports
occupied. I believe it was possible to do this within the
'init' of the module, but failed so far.
I will look into it, but if someone has a direct hint....
HA HA
0- Harrie Hazewinkel --------------------------------------0
mailto:harrie.hazewinkel@jrc.it phone:+39+332+789384
http://porto.jrc.it/~harrie/ fax:+39+332+785500
postal: JRC of the E.C. - CEO unit
Ispra 21020 (VA) Italy
0----------------------------------------------------------0
Ik ben Harrie en ik ben 28 jaar en doe SNMP na. MIB, MIB
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> I don't think so... what I'm imagining is just a function
> ap_shared_alloc(pool *p, size_t size) which is called only during init and
> returns a pointer to a piece of shared memory of size size, with cleanup
> in pool p. It should return a pointer to an ap_shared_t. Something
> like:
>
> typedef struct {
> void *mem; /* offset of first byte */
> /* possibly other stuff which is not part of the public interface */
> /* such as int fd; for the file version */
> } ap_shared_t;
>
> Each allocation would call shmget() or mmap() or whatever... so we'd
> actually have multiple segments. The core would use this to allocate the
> scoreboard.
>
> To support file-based "shared" mem you'd need
> ap_shared_get(ap_shared_t, size_t offset, size_t len) which re-reads
> the len bytes at offset. And ap_shared_put() which re-writes them.
> These should be macro'd to nothing for the non-file case.
Hmmm... well, I suppose these would be a useful first step towards
something more general purpose.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Dean Gaudet <dg...@arctic.org>.
I don't think so... what I'm imagining is just a function
ap_shared_alloc(pool *p, size_t size) which is called only during init and
returns a pointer to a piece of shared memory of size size, with cleanup
in pool p. It should return a pointer to an ap_shared_t. Something
like:
typedef struct {
void *mem; /* offset of first byte */
/* possibly other stuff which is not part of the public interface */
/* such as int fd; for the file version */
} ap_shared_t;
Each allocation would call shmget() or mmap() or whatever... so we'd
actually have multiple segments. The core would use this to allocate the
scoreboard.
To support file-based "shared" mem you'd need
ap_shared_get(ap_shared_t, size_t offset, size_t len) which re-reads
the len bytes at offset. And ap_shared_put() which re-writes them.
These should be macro'd to nothing for the non-file case.
Dean
On Tue, 20 Jan 1998, Ben Laurie wrote:
> Dean Gaudet wrote:
> > core. (Yup it would be so nice if we would abstract shared memory so you
> > don't have to do all the same crud we do. A patch for that would be
> > awesome.)
>
> I keep thinking about this one - the problem is that we need something
> more than just a shared memory abstraction, don't we? Unless you happen
> to want to store something that's fixed length and one-per-process, that
> is.
>
> Cheers,
>
> Ben.
>
> --
> Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
> Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
> and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
> A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
> London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
>
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Harrie Hazewinkel <ha...@jrc.it>.
The reasons of this kind of SNMP-logging was that every process
only has to do a small bit during the HTTP-request cycle and
the SNMP-part during a SNMP-request has to do more. But this is
justified by the fact that the number of SNMP-requests are much
less then the number of HTTP-requests. Another reason is that
the SNMP-information is similar like the status-information;
only 1 process may write in its own part and may read out of all parts.
The only thing is that when you have around 50 virtual hosts
the scorebaord becomes huge.
Since I recently wanted to implement the complete WWW-MIB I ran
into the problem that I need a sequence of the documents access
(sorted by virtual host). So I started to search for other ways
of maintaining the SNMP-information. I believe a UNIX-domain socket
is probably the best choice. Then all processes write a small
datagram and send it to the UNIX-domain socket. The SNMP-process
has then to read all the packets and log the SNMP-information.
The only drawback here is that the processing cylces go up,
but we do not have the SNMP-information in shared memeory
and in size multplied by the number of apache processes.
Dirk-Willem van Gulik wrote:
>
> On Tue, 20 Jan 1998, Ben Laurie wrote:
>
> > Dean Gaudet wrote:
> > > core. (Yup it would be so nice if we would abstract shared memory so you
> > > don't have to do all the same crud we do. A patch for that would be
> > > awesome.)
>
> > I keep thinking about this one - the problem is that we need something
> > more than just a shared memory abstraction, don't we? Unless you happen
> > to want to store something that's fixed length and one-per-process, that
> > is.
>
> I got as far as adding an extra struct to the module vector with allows
> for the registry of a 'shared_len(0 | pid) and shared_ptr(0 | pid)'
> function pointers.But unless you are a very displined programmer; it is
> easy to abuse that interface and end up with a per-process/per-vhost type
> of strorage which is sparsely used.
>
> Dw.
>
--
HA HA
0- Harrie Hazewinkel --------------------------------------0
mailto:harrie.hazewinkel@jrc.it phone:+39+332+789384
http://porto.jrc.it/~harrie/ fax:+39+332+785500
postal: JRC of the E.C. - CEO unit
Ispra 21020 (VA) Italy
0----------------------------------------------------------0
Ik ben Harrie en ik ben 28 jaar en doe SNMP na. MIB, MIB
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Dirk-Willem van Gulik <di...@jrc.it>.
On Tue, 20 Jan 1998, Ben Laurie wrote:
> Dean Gaudet wrote:
> > core. (Yup it would be so nice if we would abstract shared memory so you
> > don't have to do all the same crud we do. A patch for that would be
> > awesome.)
> I keep thinking about this one - the problem is that we need something
> more than just a shared memory abstraction, don't we? Unless you happen
> to want to store something that's fixed length and one-per-process, that
> is.
I got as far as adding an extra struct to the module vector with allows
for the registry of a 'shared_len(0 | pid) and shared_ptr(0 | pid)'
function pointers.But unless you are a very displined programmer; it is
easy to abuse that interface and end up with a per-process/per-vhost type
of strorage which is sparsely used.
Dw.
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
> core. (Yup it would be so nice if we would abstract shared memory so you
> don't have to do all the same crud we do. A patch for that would be
> awesome.)
I keep thinking about this one - the problem is that we need something
more than just a shared memory abstraction, don't we? Unless you happen
to want to store something that's fixed length and one-per-process, that
is.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Dean Gaudet <dg...@arctic.org>.
You know I think the patch to the core of the code can be eliminated with
a little extra work in 1.3.
For example, we already maintain bytes_out in BUFFs, we can easily
maintain bytes_in. It would be really useful to distinguish between
protocol overhead (i.e. headers, chunking headers/footers) and actual
content -- for example, in logs it would be nice to know how much of a
file was actually sent, and how much the header overhead was. This deals
with all the counters you've added to the buff and conn_rec structures.
I don't understand why you need to put the vhost stuff in the scoreboard.
You could implement your own shared memory and not require patches to the
core. (Yup it would be so nice if we would abstract shared memory so you
don't have to do all the same crud we do. A patch for that would be
awesome.)
Dean
On Tue, 20 Jan 1998, Harrie Hazewinkel wrote:
> Hi,
>
> I have upgraded the snmp module for the Apache 1.2.5 version.
> I also fixed some changes as done within the MIB-definition
> by the SYSAPPL-WG of the IETF and reduced memory-use for
> the SNMP-engine. Therefore, I have created a tarball which
> includes the 'patch' required in the original Apache code
> (version 1.2.5) and the 'module/snmp' directory.
>
> Info is available at
> http://www-musiq.jrc.it/~harrie/software/
> and the tarbal is at
> http://www-musiq.jrc.it/~harrie/software/mod_snmp1b2_apache1.2.5.tar.gz
>
> It is tested on Solaris2.5.1 and FreeBSD2.
>
> So far I only encountered only a problem during a 'KILL -HUP'.
> The SNMP-process is then not killed and it keeps the ports
> occupied. I believe it was possible to do this within the
> 'init' of the module, but failed so far.
> I will look into it, but if someone has a direct hint....
>
>
>
> HA HA
> 0- Harrie Hazewinkel --------------------------------------0
> mailto:harrie.hazewinkel@jrc.it phone:+39+332+789384
> http://porto.jrc.it/~harrie/ fax:+39+332+785500
> postal: JRC of the E.C. - CEO unit
> Ispra 21020 (VA) Italy
> 0----------------------------------------------------------0
> Ik ben Harrie en ik ben 28 jaar en doe SNMP na. MIB, MIB
>
Re: Upgrade of the SNMP module for Apache 1.2.5
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 20 Jan 1998, Harrie Hazewinkel wrote:
> So far I only encountered only a problem during a 'KILL -HUP'.
> The SNMP-process is then not killed and it keeps the ports
> occupied. I believe it was possible to do this within the
> 'init' of the module, but failed so far.
> I will look into it, but if someone has a direct hint....
In 1.2 you should be able to do it by using register_cleanup in the pool
passed to the init() function. Your cleanup should kill the child. That
works for certain in 1.3.
All the code from the fork() through the register_cleanup() should be
inside a block_alarms()/unblock_alarms() section.
Dean