You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Stas Bekman <st...@stason.org> on 2004/03/07 01:17:38 UTC
[Fwd: Re[2]: GIF contents not "coming out"]
Your friendly forwarding service is hard at work ;)
-------- Original Message --------
Subject: Re[2]: GIF contents not "coming out"
Date: Wed, 3 Mar 2004 21:55:21 +1100
From: Chris Drake <ch...@pobox.com>
To: Stas Bekman <st...@stason.org>
References: <69...@pobox.com>
<40...@stason.org> <36...@pobox.com>
<40...@stason.org>
Hi Stas,
Cool - glad you found it! Sorry I was so "harsh" with my $0.02 (long
night coding tends to shorten my temper - oops).
You might like to mention in your manual how to set the response code,
like "HTTP/1.1 304 NOT MODIFIED" or whatever. I noticed that you
appear to be doing some "smarts" and sending this back (in preference
to what I really printed) when you detect my etag and/or "last
modifed" headers match the request, but I'm sure other people (and me
in future) will want to set these codes - so mentioning how to do this
in the "Generating_HTTP_Response_Headers" section would be ideal.
Incadentally - there's a cosmetic bug with the code which does that
too: I find my log files full of this:-
[Mon Mar 01 00:13:54 2004] [warn] /cache.asp/blank1x1.gif did not
send an HTTP header
but upon close investigation, my script does send the correct stuff,
and the correct (well - correctED to be precise - you code stripped
out my reply and sent "304 NOT MODIFED" instead - which is correct)
headers come back to the browser just fine - it's only when the
original request had an "IF-Modified-Since:" condition that the error
turns up in the logs (even though it wasn't an error).
If you have time to experiment, I noticed a horrendous "memory leak"
*or* code "zombieing" problem using the "worker MPM" instead of
prefork with mod_perl and DBI and oracle. Problem vanished when I
changed back to prefork. Dunno if it was mod_perl, DBI, or oracle to
blame unfortunately - but I thought I'd tell you in case it helps.
I'm using RedHat enterprise linux 3 with mod_perl-1.99_09
Kind Regards,
Chris Drake
Wednesday, March 3, 2004, 7:26:20 PM, you wrote:
SB> Hi Chris,
SB> Finally I've got a chance to analyze your problem.
>> The "GIF" isn't actually a GIF at all - it was the "favicon.ico" icon
>> file - so my guess is that mod_perl is sneaking a peek at what comes
>> next (expecting GIF89a or whatever) and doing something wrong when
>> that 'aint it. I realize I shouldn't be saying "image/gif" in this
>> case, but then - mod_perl shouldn't be screwing up my headers just
>> 'coz of that mistake?
SB> mod_perl wasn't doing anything like that. The problem was in the contents of
SB> favicon.ico. .ico files seem to start with the sequence \000\000 and mod_perl
SB> 2.0 had a bug dealing with parsing printed headers immediatelly followed by
SB> response body starting with '\0'. I've reduced your test to a one liner
script:
SB> print "Content-type: image/x-icon\n\n\000\000\001\000";
SB> It's now fixed in the cvs.
>> On a personal note - I think the whole mod_perl header system should
>> have been omitted. If a script wants to send the headers, it should
>> be allowed to, without interference. Making people learn an
>> indeterminate number of new custom non-standard callbacks like
>> "content_type()" and whatever the ones are for caching, p3p, cookies,
>> accept, encoding, charsets, and so forth is silly - since everyone who
>> knows they're needed already knows how to send them normally - hence
>> the extra bunch of callback stuff just wastes everyone's time. Just
>> my $0.02...
SB> Nobody prevents you from printing your own headers. It's just much slower. I
SB> just wrote a new docs section explaining the differences and nuances of
SB> headers generating:
SB>
http://perl.apache.org/docs/2.0/user/coding/coding.html#Generating_HTTP_Response_Headers
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by "Jeffrey W. Baker" <jw...@acm.org>.
On Sun, 2004-03-07 at 13:34, Perrin Harkins wrote:
> Jeffrey W. Baker wrote:
> > I wasn't aware of these issues before. Looks like I'll have to use
> > BerkeleyDB or Tie::DB_Lock or something.
>
> On closer inspection I think it's mostly okay because the session object
> is supposed to go out of scope and get destroyed (and thus untie) after
> every interaction, and the locking should mean that only process is
> writing during that time. You could run into trouble though if a
> process is able to open for reading and then later do writes without
> closing and re-opening the file first, and it looks like that does
> happen. This could lead to lost updates from other processes.
>
> I'd suggest either looking at how MLDBM::Sync works, or using BerkeleyDB
> instead. BerkeleyDB can be tricky to get right, but it's significantly
> faster than any other storage mechanism supported by Apache::Session.
It's faster in the trivial case, but doesn't exactly scale horizontally,
that is to more than one server. I've also seen BerkeleyDB doing seek,
read, write, sync in such a pattern that the Linux kernel will starve
all other I/O on the machine. But this is a platform-specific trouble
:)
In any case, it will be a simple addition.
-jwb
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, 2004-03-08 at 13:44, Chris Ochs wrote:
> sub handler {
> $r = shift;
> untie %session;
%session is a global then? That explains why you have trouble when the
script dies. I'd suggest switching to using $r->pnotes() for storing
it. Alternatively, you could use a cleanup handler to clear it, but I
don't trust that as much the pnotes approach.
- Perrin
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Chris Ochs <ch...@paymentonline.net>.
----- Original Message -----
From: "Perrin Harkins" <pe...@elem.com>
To: "Chris Ochs" <ch...@paymentonline.net>
Cc: "Modperl List" <mo...@perl.apache.org>
Sent: Monday, March 08, 2004 6:27 AM
Subject: Re: Apache::Session and DB_File problems
> On Sun, 2004-03-07 at 21:13, Chris Ochs wrote:
> > Been doing some more testing on this. Previously I was untying the
> > sessions when the handler was finished. But it seems that all it takes
is
> > just one child to exit uncleanly where I cant' catch the error and untie
the
> > sesssions, and that's all it takes to cause corruption. If I untie the
> > session variables at the very start of the handler everything works
fine.
>
> What do you mean by untying the sessions? Can you show us some code?
> All you should need to do is make sure the session object goes out of
> scope, which will happen even if your code dies.
Here is the start and end of the handler. Also at the bottom is the code
that I use to create the session.
sub handler {
$r = shift;
untie %session;
untie %basket;
untie %cs;
...
Do stuff here...
$myapp::template->process($req_file, $vars, $r) || return fail($r, OK,
$template->error);
return success(OK);
} # end handler
sub success {
my ($status) = @_;
$session{'timestamp'} = time if (defined %session);
$basket{'timestamp'} = time if (defined %basket);
$cs{'timestamp'} = time if (defined %cs);
untie %session;
untie %basket;
untie %cs;
$myapp::logger->debug("EndRequest");
return $status;
}
sub fail {
my ($r, $status, $message) = @_;
$session{'timestamp'} = time if (defined %session);
$basket{'timestamp'} = time if (defined %basket);
$cs{'timestamp'} = time if (defined %cs);
untie %session;
untie %basket;
untie %cs;
$myapp::logger->info("Error: $message");
$params{'error_message'} = $message;
my $vars = { req => $r, uri => $r->uri, env => \%ENV, params =>
\%params, };
$Cart110::template->process("template_error.html", $vars, $r)
|| print "<h4><font color=red>Error: </font>$message</h4>";
$r->log_reason($message, $r->filename);
$myapp::logger->debug("EndRequest (ERROR=$message)");
return $status;
}
Session code:
sub basket {
my $dbfile = "$dbdir/$catalog" . "_basket.db";
if ($params{'basket'}) {
$session{'basket'} = $params{'basket'};
}
eval {
tie %basket, 'Apache::Session::DB_File', $session{'basket'}, {
FileName => "$dbfile",
LockDirectory => "$dbdir/lock",
};
};
if ($@) {
tie %basket, 'Apache::Session::DB_File', undef, {
FileName => "$dbfile",
LockDirectory => "$dbdir/lock",
};
}
$session{'basket'} = $basket{_session_id};
$Cart110::logger->debug("BasketID = $basket{_session_id}");
}
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Perrin Harkins <pe...@elem.com>.
On Sun, 2004-03-07 at 21:13, Chris Ochs wrote:
> Been doing some more testing on this. Previously I was untying the
> sessions when the handler was finished. But it seems that all it takes is
> just one child to exit uncleanly where I cant' catch the error and untie the
> sesssions, and that's all it takes to cause corruption. If I untie the
> session variables at the very start of the handler everything works fine.
What do you mean by untying the sessions? Can you show us some code?
All you should need to do is make sure the session object goes out of
scope, which will happen even if your code dies.
> As far as performance, I would say any db file implementation would be
> slower if you are using lock files, because you can't have concurrent access
> on the same file. The file store is about 8 times as fast as DB_File in my
> setup.
BerkeleyDB handles its own locking and thus does not require any lock
files.
> BerkeleyDB is fast, but in concurrent data store mode which is the easiest
> to use when you want concurrent access, you still have to write to disk on
> every access to flush the memory cache to disk or other processes won't see
> the latest writes to the database. At least that has been my limited
> experience anyways.
That hasn't been my experience. I've never needed to do anything
explicit in order to get the data to be shared.
- Perrin
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Chris Ochs <ch...@paymentonline.net>.
>
> On closer inspection I think it's mostly okay because the session object
> is supposed to go out of scope and get destroyed (and thus untie) after
> every interaction, and the locking should mean that only process is
> writing during that time. You could run into trouble though if a
> process is able to open for reading and then later do writes without
> closing and re-opening the file first, and it looks like that does
> happen. This could lead to lost updates from other processes.
>
> I'd suggest either looking at how MLDBM::Sync works, or using BerkeleyDB
> instead. BerkeleyDB can be tricky to get right, but it's significantly
> faster than any other storage mechanism supported by Apache::Session.
>
> The corruption problems in this thread could also have been caused by
> the usual suspect -- scoping issues. If the session object doesn't get
> destroyed, it will keep the file open, and that is definitely not safe.
>
> - Perrin
Been doing some more testing on this. Previously I was untying the
sessions when the handler was finished. But it seems that all it takes is
just one child to exit uncleanly where I cant' catch the error and untie the
sesssions, and that's all it takes to cause corruption. If I untie the
session variables at the very start of the handler everything works fine.
As far as performance, I would say any db file implementation would be
slower if you are using lock files, because you can't have concurrent access
on the same file. The file store is about 8 times as fast as DB_File in my
setup.
BerkeleyDB is fast, but in concurrent data store mode which is the easiest
to use when you want concurrent access, you still have to write to disk on
every access to flush the memory cache to disk or other processes won't see
the latest writes to the database. At least that has been my limited
experience anyways.
Chris
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Perrin Harkins <pe...@elem.com>.
Jeffrey W. Baker wrote:
> I wasn't aware of these issues before. Looks like I'll have to use
> BerkeleyDB or Tie::DB_Lock or something.
On closer inspection I think it's mostly okay because the session object
is supposed to go out of scope and get destroyed (and thus untie) after
every interaction, and the locking should mean that only process is
writing during that time. You could run into trouble though if a
process is able to open for reading and then later do writes without
closing and re-opening the file first, and it looks like that does
happen. This could lead to lost updates from other processes.
I'd suggest either looking at how MLDBM::Sync works, or using BerkeleyDB
instead. BerkeleyDB can be tricky to get right, but it's significantly
faster than any other storage mechanism supported by Apache::Session.
The corruption problems in this thread could also have been caused by
the usual suspect -- scoping issues. If the session object doesn't get
destroyed, it will keep the file open, and that is definitely not safe.
- Perrin
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by "Jeffrey W. Baker" <jw...@acm.org>.
On Sun, 2004-03-07 at 09:31, Perrin Harkins wrote:
> Chris Ochs wrote:
> > This must be my day... I am stress testing a new application that uses
> > apache::session with the DB_File backend store. When hitting it with apache
> > bench after a few hundred connections something with the db file gets
> > wacked, and apache starts to segfault until I delete the .db file, at which
> > point everything is ok again.
>
> The code in Apache::Session::Store::DB_File does not look safe to me.
> It doesn't untie the file between writes, which is a problem with
> DB_File because of the built-in caching it uses. If you just want safe
> access to DB_File from multiple processes, you should try MLDBM::Sync
> instead. If you want to use Apache::Session, switch to the MySQL or
> file storage instead.
I wasn't aware of these issues before. Looks like I'll have to use
BerkeleyDB or Tie::DB_Lock or something. Thanks for the tip.
-jwb
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Perrin Harkins <pe...@elem.com>.
Chris Ochs wrote:
> This must be my day... I am stress testing a new application that uses
> apache::session with the DB_File backend store. When hitting it with apache
> bench after a few hundred connections something with the db file gets
> wacked, and apache starts to segfault until I delete the .db file, at which
> point everything is ok again.
The code in Apache::Session::Store::DB_File does not look safe to me.
It doesn't untie the file between writes, which is a problem with
DB_File because of the built-in caching it uses. If you just want safe
access to DB_File from multiple processes, you should try MLDBM::Sync
instead. If you want to use Apache::Session, switch to the MySQL or
file storage instead.
- Perrin
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Session and DB_File problems
Posted by Ged Haywood <ge...@jubileegroup.co.uk>.
Hi there,
On Sat, 6 Mar 2004, Chris Ochs wrote:
> stress testing a new application that uses apache::session with the
> DB_File backend store. When hitting it with apache bench after a
> few hundred connections something with the db file gets wacked, and
> apache starts to segfault until I delete the .db file, at which
> point everything is ok again.
Are you sure it's something within Apache itself that's segfaulting?
73,
Ged.
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Apache::Session and DB_File problems
Posted by Chris Ochs <ch...@paymentonline.net>.
This must be my day... I am stress testing a new application that uses
apache::session with the DB_File backend store. When hitting it with apache
bench after a few hundred connections something with the db file gets
wacked, and apache starts to segfault until I delete the .db file, at which
point everything is ok again. This is apache 1.3.29/mod perl 1.29, perl
5.6.1, freebsd 5.2.1.
Chris
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::Reload cause memory leaks?
Posted by Perrin Harkins <pe...@elem.com>.
Chris Ochs wrote:
> When I have apache::reload as an init handler in httpd.conf, apache
> processes will randomly go from 20mb to 120 or sometimes over 500 mb of
> memory. I haven't narrowed it down to an exact place in the code that
> triggers it, because simply taking out apache::reload made it stop.
I suppose this could happen if all of your code was reloaded in a
process. I'm not sure why it would do that though. If you want to see
what's being reloaded add "PerlSetVar ReloadDebug On" to your httpd.conf
and watch the error_log.
- Perrin
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Apache::Reload cause memory leaks?
Posted by Chris Ochs <ch...@paymentonline.net>.
I have a really strange problem and although I'm not 100% sure it's due to
Apache::Reload, it seems the most likely candidate. This is apache 1.3.29
and mod perl 1.29 with perl 5.6.1 on freebsd 5.2.1. My code is a perl
handler and with all the modules preloaded via startup.pl each apache
process uses right at 20mb of memory.
When I have apache::reload as an init handler in httpd.conf, apache
processes will randomly go from 20mb to 120 or sometimes over 500 mb of
memory. I haven't narrowed it down to an exact place in the code that
triggers it, because simply taking out apache::reload made it stop.
Sometimes I have to run the code dozens of times to trigger the memory
usage, other times it happens on the first try.
In httpd.conf this is how I had apache::reload:
PerlInitHandler Apache::Reload
PerlSetVar ReloadAll Off
I only had 'use Apache::Reload' in one very small module in my code.
Anyone heard of this type of problem before with Apache::Reload?
Chris
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html