You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Bob Foster <bf...@hotresponse.com> on 2001/07/24 21:33:45 UTC
Increasing Shared Memory
Hi,
I'm using Stas Bekman's excellent Apache::VMonitor module to help me
decrease my mod_perl child process memory usage. I was working on
preloading all of my perl modules and scripts in a startup.pl script when
I noticed that the amount of shared memory seemed very low. Immediately
after I make an Oracle database connection, the child jumps from a size of
3.6M (2.4M shared) to 17.4M (3.4M shared). The child process slowly grows
to 22.2M (3.4M shared). The loaded libs Sizes total 13.6M.
Shouldn't the libs load into shared memory? If so, how?
Here's a snippet from Apache::VMonitor.
Loaded libs Sizes: (in bytes)
5800677 ( 5.5M) : /ora1/oracle/product/8.1.5/lib/libclntsh.so.8.0
4118299 ( 3.9M) : /lib/libc-2.1.2.so
1134094 ( 1.1M) : /usr/local/apache_perl/bin/httpd_perl
540120 ( 527k) : /lib/libm-2.1.2.so
372604 ( 364k) : /lib/libnsl-2.1.2.so
344890 ( 337k) : /lib/ld-2.1.2.so
254027 ( 248k) : /lib/libnss_nis-2.1.2.so
253826 ( 248k) : /lib/libnss_nisplus-2.1.2.so
247381 ( 242k) : /lib/libpthread-0.8.so
247348 ( 242k) : /lib/libnss_files-2.1.2.so
162740 ( 159k) : /usr/lib/libglib-1.2.so.0.0.5
81996 ( 80k) :
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/Oracle/Oracle.so
74663 ( 73k) : /lib/libdl-2.1.2.so
64595 ( 63k) : /lib/libcrypt-2.1.2.so
63848 ( 62k) : /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.so
60026 ( 59k) : /usr/lib/libgtop_sysdeps.so.1.0.2
58088 ( 57k) :
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Date/Calc/Calc.so
51146 ( 50k) :
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Storable/Storable.so
48500 ( 47k) : /usr/lib/perl5/site_perl/5.005/i386-linux/auto/GTop/GTop.so
46795 ( 46k) : /usr/local/apache_perl/libexec/mod_include.so
32545 ( 32k) : /usr/lib/gconv/ISO8859-1.so
31927 ( 31k) : /usr/lib/libgtop.so.1.0.2
29970 ( 29k) : /usr/share/locale/en_US/LC_COLLATE
28657 ( 28k) : /usr/lib/libgdbm.so.2.0.0
27405 ( 27k) : /usr/local/apache_perl/libexec/mod_status.so
24409 ( 24k) :
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Apache/Scoreboard/Scoreboard.so
22970 ( 22k) :
/usr/lib/perl5/5.00503/i386-linux/auto/Data/Dumper/Dumper.so
17732 ( 17k) : /usr/lib/libgtop_common.so.1.0.2
14748 ( 14k) : /usr/lib/perl5/5.00503/i386-linux/auto/IO/IO.so
11137 ( 11k) :
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Time/HiRes/HiRes.so
10764 ( 11k) : /usr/local/apache_perl/libexec/mod_usertrack.so
10428 ( 10k) : /usr/share/locale/en_US/LC_CTYPE
9827 ( 10k) : /usr/lib/perl5/5.00503/i386-linux/auto/Fcntl/Fcntl.so
4847 ( 5k) : /ora1/oracle/product/8.1.5/lib/libskgxp8.so
508 ( 1k) : /usr/share/locale/en_US/LC_TIME
93 ( 1k) : /usr/share/locale/en_US/LC_MONETARY
44 ( 1k) : /usr/share/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
27 ( 1k) : /usr/share/locale/en_US/LC_NUMERIC
Thanks for your help!
Bob Foster
Re: Increasing Shared Memory
Posted by pe...@elem.com.
Quoting Joshua Chamas <jo...@chamas.com>:
> Also, more a side note, I have found that you have to fully
> restart apache, not just a graceful, if either the Oracle server
> is restarted or the TNS listener is restarted.
We fixed this at eToys by having children that failed to connect to the
database exit after the current request. Newly spawned children seem to
connect okay.
- Perrin
Re: Increasing Shared Memory
Posted by Joshua Chamas <jo...@chamas.com>.
Bob Foster wrote:
>
> Thank you very much, Joshua. I have made some progress and am now seeing
> 15.8M shared out of 16.7M on the parent. I believe that the problem was
> that I was doing a graceful restart which wasn't restarting the parent
> process.
>
> Now I have a different problem. When I initially start apache, everything
> is OK, but when the child processes die (due to MaxRequestsPerChild or me
> killing them) , the new child doesn't have a database connection and I'm
> getting "Internal Server Error." I suspect I can sort that mess out after
> a bit more troubleshooting.
>
Cool, but each child will need its own database connection, and
last time I worked with Oracle like this, it was a minimum 3M
unshared just for the Oracle connection. So just make sure
that each child httpd is doing its own DBI->connect() and not
reusing a handle created in the parent httpd or you will have
problems.
Also, more a side note, I have found that you have to fully
restart apache, not just a graceful, if either the Oracle server
is restarted or the TNS listener is restarted.
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks Founder Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re(2): Increasing Shared Memory
Posted by Bob Foster <bf...@hotresponse.com>.
joshua@chamas.com writes:
>
>Make sure to "use DBD::Oracle" in your startup.pl or
>do PerlModule DBD::Oracle ... that should load up some
>Oracle libs in the parent. Also, you *might* try
>doing a connect or even an invalid connect to Oracle,
>which might grab some extra libs that it only loads
>at runtime.
>
>If you do a connect in the parent httpd to Oracle, make sure
>that Apache::DBI doesn't cache it later for the children,
>which is why I was suggesting doing an invalid connect
>with user/pass like "BADBAD/BADBAD"
>
Thank you very much, Joshua. I have made some progress and am now seeing
15.8M shared out of 16.7M on the parent. I believe that the problem was
that I was doing a graceful restart which wasn't restarting the parent
process.
Now I have a different problem. When I initially start apache, everything
is OK, but when the child processes die (due to MaxRequestsPerChild or me
killing them) , the new child doesn't have a database connection and I'm
getting "Internal Server Error." I suspect I can sort that mess out after
a bit more troubleshooting.
Thanks again!
Bob Foster
Re: Increasing Shared Memory
Posted by Joshua Chamas <jo...@chamas.com>.
Bob Foster wrote:
>
> Hi,
>
> I'm using Stas Bekman's excellent Apache::VMonitor module to help me
> decrease my mod_perl child process memory usage. I was working on
> preloading all of my perl modules and scripts in a startup.pl script when
> I noticed that the amount of shared memory seemed very low. Immediately
> after I make an Oracle database connection, the child jumps from a size of
> 3.6M (2.4M shared) to 17.4M (3.4M shared). The child process slowly grows
> to 22.2M (3.4M shared). The loaded libs Sizes total 13.6M.
>
> Shouldn't the libs load into shared memory? If so, how?
>
Make sure to "use DBD::Oracle" in your startup.pl or
do PerlModule DBD::Oracle ... that should load up some
Oracle libs in the parent. Also, you *might* try
doing a connect or even an invalid connect to Oracle,
which might grab some extra libs that it only loads
at runtime.
If you do a connect in the parent httpd to Oracle, make sure
that Apache::DBI doesn't cache it later for the children,
which is why I was suggesting doing an invalid connect
with user/pass like "BADBAD/BADBAD"
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks Founder Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Increasing Shared Memory
Posted by pe...@elem.com.
Quoting Bob Foster <bf...@hotresponse.com>:
> Immediately
> after I make an Oracle database connection, the child jumps from a size
> of
> 3.6M (2.4M shared) to 17.4M (3.4M shared). The child process slowly
> grows
> to 22.2M (3.4M shared). The loaded libs Sizes total 13.6M.
>
> Shouldn't the libs load into shared memory? If so, how?
DBI->install_driver()
See http://perl.apache.org/guide/performance.html#Initializing_DBI_pm for more.
- Perrin