mp2/perl bug in substr( lc( $utf8_url ) )

[John ORourke <jo...@o-rourke.org>]

Hi folks,

I appear to have found a bug, possibly in perl's utf8 handling but I 
need to create a test to make sure.  I have a workaround which is 
surprisingly simple, hopefully someone can explain it!

I'm not using the 'mp2bug' template because I have 2 different 
architectures, 1 which exhibits the bug and one which does not - they 
are described at the bottom.

I'm going to try the obvious solution - upgrading Apache and libapreq to 
see if that fixes it, when I have some time.

Anyway here it is: I was doing this in a handler: (abbreviated for 
simplicity)

-------------------------------------apache2.conf:

PerlTypeHandler $My::Persistent::Object->type_handler

-------------------------------------My/Persistent/Object.pm:

sub type_handler {  # method handler for Type phase, with a persistent 
object
    my $r=shift;
    my $uri=decode_utf8($r->uri());
    warn "got uri $uri\n";
    my $path=substr(lc($uri),0,2048); # limit ridiculous paths
    warn "got path $path";
}

-------------------------------------------------------

After a given process had served a few requests I was seeing this:
    got uri /something/here
    got path /somethi   <--- path truncated to a certain no. of chars

Then it got stranger - it seemed to truncate all URLs sent to that 
process to that same no. of chars for the life of the process, and the 
no. of chars appeared to be a low but random number - 6, 15 etc!  
Clearly by this point I was going slightly insane so these assumptions 
may not be entirely
correct.

----------------------------------------------WORKAROUND:
Then even stranger - by replacing this:
    my $path = substr( lc( $uri ), 0, 2048 )
with
    my $path = lc( $uri );
    $path=substr( $path, 0, 2048 );
the problem went away!

Oddly, I have seen this on one setup but not another....

Here is the setup I DID see the problem on:

-------------------------------------------
OS: Ubuntu 6.06.1 LTS
Apache: 2.0.55 prefork mpm, 32-bit
Perl: 5.8.7  (also seen on a similar setup with 5.8.8)
Apache2::Request   : 2.07
CGI                : 3.10
ExtUtils::MakeMaker: 6.17, 6.38
LWP                : 5.803
mod_perl           : -
mod_perl2          : 2.000002
-----------------------------------------------

And here is the one I DID NOT see it on:
-----------------------------------------------
Fedora Core 5
Perl 5.8.8
Linux 2.6.9-34.elsmp
Apache 2.2.0 prefork mpm, 32-bit
Apache2::Request   : 2.08
CGI                : 3.15
ExtUtils::MakeMaker: 6.30
LWP                : 5.805
mod_perl           : -
mod_perl2          : 2.000002

----------------------------------------------

For now I don't have time to investigate further, having found a 
workaround, but I'll post again if I find anything more.

cheers
John