You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Greg Balmer <Gr...@Mapflow.com> on 2004/04/13 19:40:42 UTC
MP2 Underrun Filter (No EOS it seems!)
Hi All,
I'm writing a connection filter handler and have started with the example
underrun filter from the mp2 doc's.
I basically want to read all the data from the bucket brigades until I
receive the EOS and then process an entire request
(POST'ed XML Data in my case). The Problem is that I never seem to receive
the EOS bucket! and if this never occurs
then I don't know if the request is complete and I can't send an EOS through
to the next filter so the server blocks!
Has anyone had a similar problem? or is there a mistake in the example? I'm
too new to this to be able to spot it if there is!
I have included the filter I'm working with below.
Many thanks in advance,
Greg.
P.s BTW: I'm runing apache 2.0.49 (MPM worker), latest stable MP2, perl
5.8.3 (with threads) on solaris 8
use strict;
use warnings;
use base qw(Apache::Filter);
use APR::Brigade ();
use APR::Bucket ();
use Apache::Const -compile => qw(MODE_READBYTES OK M_POST);
use APR::Const -compile => qw(SUCCESS BLOCK_READ);
use Apache::Const -compile => qw(OK M_POST);
use Apache::Connection ();
use constant TOKEN_SIZE => 300; # 300bytes
sub handler : FilterConnectionHandler {
my($f, $bb, $mode, $block, $readbytes) = @_;
my $ba = $f->c->bucket_alloc;
my $ctx = $f->ctx;
my $buffer = defined $ctx ? $ctx : '';
$ctx = ''; # reset
my $seen_eos = 0;
my $data;
warn "\nfilter called\n";
# fetch and consume bucket brigades untill we have at least TOKEN_SIZE
# bytes to work with
do {
my $tbb = APR::Brigade->new($f->c->pool, $ba);
my $rv = $f->next->get_brigade($tbb, $mode, $block, $readbytes);
warn "asking for a bb\n";
($data, $seen_eos) = flatten_bb($tbb);
$tbb->destroy;
$buffer .= $data;
} while (!$seen_eos && length($buffer) < TOKEN_SIZE);
# now create a bucket per chunk of TOKEN_SIZE size and put the remainder
# in ctx
for (split_buffer($buffer)) {
if (length($_) == TOKEN_SIZE) {
$bb->insert_tail(APR::Bucket->new($_));
}
else {
$ctx .= $_;
}
}
my $len = length($ctx);
if ($seen_eos) {
# flush the remainder
$bb->insert_tail(APR::Bucket->new($ctx));
$bb->insert_tail(APR::Bucket::eos_create($ba));
warn "seen eos, flushing the remaining: $len bytes\n";
}
else {
# will re-use the remainder on the next invocation
$f->ctx($ctx);
warn "storing the remainder: $len bytes\n";
}
return Apache::OK;
}
# split a string into tokens of TOKEN_SIZE bytes and a remainder
sub split_buffer {
my $buffer = shift;
if ($] < 5.007) {
my @tokens = $buffer =~ /(.{@{[TOKEN_SIZE]}}|.+)/g;
return @tokens;
}
else {
# available only since 5.7.x+
return unpack "(A" . TOKEN_SIZE . ")*", $buffer;
}
}
sub flatten_bb {
my ($bb) = shift;
my $seen_eos = 0;
my @data;
for (my $b = $bb->first; $b; $b = $bb->next($b)) {
if ($b->is_eos){
$seen_eos++;
last;
warn "\n ************ SEEN EOS GB";
}
$b->read(my $bdata);
$bdata = '' unless defined $bdata;
push @data, $bdata;
}
return (join('', @data), $seen_eos);
}
1;
Greg Balmer
Mapflow
4 Merrion Square
Dublin 2
IRELAND
Tel: +353-1-6341430
Fax: +353-1-6760504
E-mail: Greg.Balmer@Mapflow.com
Check out the most recent developments on our web-site:
http://www.mapflow.com
"The information in this email is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this email by
anyone else is unauthorised. If you are not the intended recipient, any
disclosure, copyright, distribution or any action taken or omitted to be
taken in reliance on it, is prohibited and may be unlawful"
[mp2] [solved] Re: Server fails to warm up during make test
Posted by Fred Moyer <fr...@taperfriendlymusic.org>.
On Apr 13, 2004, at 12:40 PM, Fred Moyer wrote:
> Make test and 't/TEST -start' is timing out for me on Darwin 7.3.0,
> mp1_99_13, Perl 5.8.3, Apache 2.0.49. I've dug around on the list and
> the site but maybe I've missed something simple here. Advice on the
> next step in determining the problem or even the solution is much
> appreciated.
I got past this by using the cvs version of mp2. Doh!
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Server fails to warm up during make test
Posted by Fred Moyer <fr...@taperfriendlymusic.org>.
Make test and 't/TEST -start' is timing out for me on Darwin 7.3.0,
mp1_99_13, Perl 5.8.3, Apache 2.0.49. I've dug around on the list and
the site but maybe I've missed something simple here. Advice on the
next step in determining the problem or even the solution is much
appreciated.
-------------8<---------- Start Bug Report ------------8<----------
1. Problem Description:
<make test>
forbin:~/dev/src/modperl/mod_perl-1.99_13 fredmoyer$ t/TEST -start
[warning] setting ulimit to allow core files
ulimit -c unlimited; /Users/fredmoyer/dev/perl-5.8.3/bin/perl
/Users/fredmoyer/dev/src/modperl/mod_perl-1.99_13/t/TEST -start
/Users/fredmoyer/dev/httpd-2.0.49_mp-1.99.13/bin/httpd -d
/Users/fredmoyer/dev/src/modperl/mod_perl-1.99_13/t -f
/Users/fredmoyer/dev/src/modperl/mod_perl-1.99_13/t/conf/httpd.conf -D
APACHE2
using Apache/2.0.49 (prefork MPM)
waiting 120 seconds for server to start: .[Tue Apr 13 12:30:24 2004]
[info] 28 Apache:: modules loaded
[Tue Apr 13 12:30:24 2004] [info] 5 APR:: modules loaded
[Tue Apr 13 12:30:24 2004] [info] base server + 16 vhosts ready to run
tests
waiting 120 seconds for server to start: ok (waited 1 secs)
server localhost:8529 started
server localhost:8530 listening (TestModperl::merge)
...
still waiting for server to warm up:
........................................................................
..................................................
the server is down, giving up after 121 secs
[ error] failed to start server! (please examine t/logs/error_log)
</make test>
<log output>
forbin:~/dev/src/modperl/mod_perl-1.99_13/t/logs fredmoyer$ tail -F
error_log
END in modperl_extra.pl, pid=11648
[Tue Apr 13 12:30:25 2004] [notice] Apache/2.0.49 (Unix)
mod_perl/1.99_13 Perl/v5.8.3 configured -- resuming normal operations
[Tue Apr 13 12:30:25 2004] [info] Server built: Apr 13 2004 11:25:15
[Tue Apr 13 12:30:25 2004] [debug] prefork.c(955): AcceptMutex:
posixsem (default: posixsem)
</log output>
2. Used Components and their Configuration:
*** mod_perl version 1.9913
*** using lib/Apache/BuildConfig.pm
*** Makefile.PL options:
MP_APXS =>
/Users/fredmoyer/dev/httpd-2.0.49_mp-1.99.13/bin/apxs
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_INST_APACHE2 => 1
MP_LIBNAME => mod_perl
MP_USE_DSO => 1
MP_USE_STATIC => 1
*** /Users/fredmoyer/dev/httpd-2.0.49_mp-1.99.13/bin/httpd -V
Server version: Apache/2.0.49
Server built: Apr 13 2004 11:25:15
Server's Module Magic Number: 20020903:7
Architecture: 32-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_POSIXSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT="/Users/fredmoyer/dev/httpd-2.0.49_mp-1.99.13"
-D SUEXEC_BIN="/Users/fredmoyer/dev/httpd-2.0.49_mp-1.99.13/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
*** /Users/fredmoyer/dev/perl-5.8.3/bin/perl -V
Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration:
Platform:
osname=darwin, osvers=7.3.0, archname=darwin-2level
uname='darwin forbin.redhotpenguin.com 7.3.0 darwin kernel version
7.3.0: fri mar 5 14:22:55 pst 2004;
root:xnuxnu-517.3.15.obj~4release_ppc power macintosh powerpc '
config_args='-Dprefix=~/dev/perl-5.8.3'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-pipe -fno-common -DPERL_DARWIN -no-cpp-precomp
-fno-strict-aliasing -I/usr/local/include',
optimize='-Os',
cppflags='-no-cpp-precomp -pipe -fno-common -DPERL_DARWIN
-no-cpp-precomp -fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='3.3 20030304 (Apple Computer, Inc. build
1495)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='
-L/usr/local/lib'
libpth=/usr/local/lib /usr/lib
libs=-ldbm -ldl -lm -lc
perllibs=-ldl -lm -lc
libc=/usr/lib/libc.dylib, so=dylib, useshrplib=false,
libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dyld.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup
-L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under darwin
Compiled at Apr 13 2004 11:03:36
%ENV:
PERL5LIB="/sw/lib/perl5:/sw/lib/perl5"
PERL_LWP_USE_HTTP_10="1"
PERL_ROOT="/Users/fredmoyer/dev/perl"
@INC:
/sw/lib/perl5
/sw/lib/perl5
/Users/fredmoyer/dev/perl-5.8.3/lib/5.8.3/darwin-2level
/Users/fredmoyer/dev/perl-5.8.3/lib/5.8.3
/Users/fredmoyer/dev/perl-5.8.3/lib/site_perl/5.8.3/darwin-2level
/Users/fredmoyer/dev/perl-5.8.3/lib/site_perl/5.8.3
/Users/fredmoyer/dev/perl-5.8.3/lib/site_perl
.
*** Packages of interest status:
Apache::Request: -
CGI : 3.01
LWP : -
mod_perl : 1.9913
3. This is the core dump trace: (if you get a core dump):
[CORE TRACE COMES HERE]
This report was generated by t/REPORT on Tue Apr 13 19:25:53 2004 GMT.
-------------8<---------- End Bug Report --------------8<----------
Note: Complete the rest of the details and post this bug report to
dev <at> perl.apache.org. To subscribe to the list send an empty
email to dev-subscribe@perl.apache.org.
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: MP2 Underrun Filter (No EOS it seems!)
Posted by Stas Bekman <st...@stason.org>.
Greg Balmer wrote:
> Hi All,
>
> I'm writing a connection filter handler and have started with the example
> underrun filter from the mp2 doc's.
> I basically want to read all the data from the bucket brigades until I
> receive the EOS and then process an entire request
> (POST'ed XML Data in my case). The Problem is that I never seem to receive
> the EOS bucket! and if this never occurs
> then I don't know if the request is complete and I can't send an EOS through
> to the next filter so the server blocks!
>
> Has anyone had a similar problem? or is there a mistake in the example? I'm
> too new to this to be able to spot it if there is!
Unfortunately Apache doesn't give you an EOS bucket on the connection level
input filters. Take a look at the debugging module's output:
http://perl.apache.org/docs/2.0/user/handlers/filters.html#All_in_One_Filter
Do you have to use a connection filter and not HTTP request one? The latter
gets the EOS bucket.
I realize the docs aren't too clear on that and need to be fixed.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html