Ongoing problem with random "Handler Not Found"

["Brendan W. McAdams" <b....@themunicenter.com>]

I've got an existing production system which I've built and maintained
over the past 3 years - I've never had a problem with mod_perl until
about 4 months ago.


During that time I built a new module - PostBlotter.pm, which was not
preloaded in startup.pl (I don't typically preload most of our periphery
applications).  We have about 90% of our 20 modules in a non-preload
state, and have never had a problem.

However, with this project, PostBlotter and ONLY PostBlotter began
exhibiting random (maybe 4 out of 10 requests?) 500 errors  - with an
error log note of:
Undefined subroutine
&Apache::MuniCenter::MarkupMatrix::PostBlotter::handler called.

And no further information.  The cause of this could not be determined,
and it was found that setting it to preload in startup.pl completely
resolved the problem. 
 
Now, a few months later, I've written yet another not-for-preload,
periphery process, and I'm seeing the exact same problem, again about 4
out of 10 times:
[Mon Nov 17 10:46:43 2003] [error] Undefined subroutine
&Apache::MuniCenter::MarkupMatrix::TaxSwap::handler called.

Unfortunately, preloading this was not possible - we found that
Inline::Java, which we load in for various bond calculation routines,
does not behave properly when preloaded, and causes the entire mod_perl
system to be unstable; consequently we can't preload this module.

At this point I need to know WHY mod_perl randomly can't find the
handler, which is properly written, runs through perl -c cleanly, under
strict and warnings, and has a 1; at the bottom.

Are there any known issues that could cause this?  Any easy way to
produce a trace of where Apache is looking and what's happening
(preferably not a recompile of apache - a trace perl module I can load
would be better).

This problem exhibits across my development, test/integration, staging,
and production environments so it's not a single box issue.

 Perl version v5.8.1 for Apache/1.3.29 (Unix) mod_perl/1.29
mod_ssl/2.8.16 OpenSSL/0.9.7c

Note this problem occurred under Perl 5.6/5.6.1 and Apache 1.3.26,
mod_perl 1.26 as well.




-- 
Brendan W. McAdams             | b.w.mcadams@themunicenter.com
Senior Applications Developer  | (646) 375-1140
TheMuniCenter, LLC             | www.themunicenter.com

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html

Re: Ongoing problem with random "Handler Not Found"

[Perrin Harkins <pe...@elem.com>]

On Mon, 2003-11-17 at 11:02, Brendan W. McAdams wrote:
> We have about 90% of our 20 modules in a non-preload
> state, and have never had a problem.

Depending on how likely these are to get called and how much they
increase the memory footprint, you might save significant amounts of RAM
by preloading them where possible (i.e. avoiding Inline::Java ones).  

> However, with this project, PostBlotter and ONLY PostBlotter began
> exhibiting random (maybe 4 out of 10 requests?) 500 errors  - with an
> error log note of:
> Undefined subroutine
> &Apache::MuniCenter::MarkupMatrix::PostBlotter::handler called.

This is just getting called directly by mod_perl when you call a URL
that is mapped to it?

> At this point I need to know WHY mod_perl randomly can't find the
> handler, which is properly written, runs through perl -c cleanly, under
> strict and warnings, and has a 1; at the bottom.

One possible answer is that @INC is different at that time, but it's
hard to find out since the call is being made from C.

> Are there any known issues that could cause this?  Any easy way to
> produce a trace of where Apache is looking and what's happening
> (preferably not a recompile of apache - a trace perl module I can load
> would be better).

If you can reproduce it reliably, you can use strace.  Another low-tech
approach would be to make a simple handler like this:

package PostBlotterLoader;

use strict;
use warnings;

sub handler {
    eval {
        require Apache::MuniCenter::MarkupMatrix::PostBlotter;
        &Apache::MuniCenter::MarkupMatrix::PostBlotter::handler;
    };
    if ($@) {
        warn "load failure: @INC is " . @INC . " and %INC is " . %INC;
        # maybe try again here?
    }
}

You can preload that handler, and map the URLs that should go to
PostBlotter to this instead.

- Perrin


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html