You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stefan Eissing <st...@greenbytes.de> on 2015/07/09 13:46:25 UTC

test base line

I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.

I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as 
"t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
"
My perl is the default Ubuntu 14.04 perl 5.18.

Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...

Thanks for the help,

  Stefan

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Re: test base line

Posted by Michael Felt <ma...@gmail.com>.
On 2015-07-09 1:46 PM, Stefan Eissing wrote:
> I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.
>
> I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as
> "t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
> Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
> "
> My perl is the default Ubuntu 14.04 perl 5.18.
What I have done - using a MUCH older version of perl is the following:

# perl -MCPAN -e shell

And once configured to get sources,

install Bundle::ApacheTest

You may run into other issues you need to resolve before the Bundle 
installs successfully.

Hope this helps!
>
> Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...
>
> Thanks for the help,
>
>    Stefan
>
> <green/>bytes GmbH
> Hafenweg 16, 48155 Münster, Germany
> Phone: +49 251 2807760. Amtsgericht Münster: HRB5782
>
>
>


Re: test base line

Posted by Stefan Eissing <st...@greenbytes.de>.
OK. Case solved. test/framework/trunk found hardly innocent:

On my Parallels Ubuntu image, the lines

127.0.0.1     localhost
::1	      localhost

existed. But the httpd.conf which is generated by t/TEST does a

Listen 0.0.0.0:8529

which is a mix of ipv4/ipv6 that causes the startup to wait
indefinitely for localhost to warm up. I assume the checks
are done on ::1 and never answered.

Removing the ::1 line from /etc/hosts solved the problem.

On the other hand, If one replaces the "Listen 0.0.0.0:port" 
with "Listen port" and adds the ::1 line again to /etc/hosts,
the startup problems are also gone. 

But the access.t tests now fail.

So, something has to give. I think the tests should work
in the presence of ipv6 nowadays. I definitely think that
the Listen should be done with port only.

//Stefan

PS. This was a plain vanilla ubuntu 14.04 lts install. nothing fancy added.



> Am 13.07.2015 um 22:17 schrieb Rainer Jung <ra...@kippdata.de>:
> 
> Am 13.07.2015 um 17:34 schrieb Stefan Eissing:
>> Ok, narrowing it down, but not done yet. Problem is as follows:
>> 
>> 1 with 'localhost' as servername, tests were failing with "waiting for server to warm up", although server was reachable as http://localhost:8529/
>>   strange.
>> 2 with 'test.example.org' as servername and entry in /etc/hosts everything works except modules/access.t with 31 failures.
>>   After reading that code, I see that
>>   a) mod_access_compat makes a reverse lookup if it thinks the deny/allow is a host name and lookups are allowed.
>>   b) HostnameLookups is 'Off' in the test config
>>   c) /etc/hosts does not enable reverse lookups for the resolver (at least not on ubuntu/osx)
>> 
>> So, it seems to me you all are testing with "127.0.0.1" or enable lookups and have a name from a DNS zone with proper reverse. Everything else should not work.
> 
> Testing with localhost here.
> 
> No precise idea here about your problem, but
> 
> - make sure you don't have any proxy shell environment variables set (http_proxy, https_proxy or similar)
> 
> - we might get an idea if we can see your /etc/hosts and /etc/resolv.conf (cleared from private info).
> 
>> //Stefan (needs a drink)
> 
> Taming the beast makes one thirsty.
> 
> Rainer

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Re: test base line

Posted by Rainer Jung <ra...@kippdata.de>.
Am 13.07.2015 um 17:34 schrieb Stefan Eissing:
> Ok, narrowing it down, but not done yet. Problem is as follows:
>
> 1 with 'localhost' as servername, tests were failing with "waiting for server to warm up", although server was reachable as http://localhost:8529/
>    strange.
> 2 with 'test.example.org' as servername and entry in /etc/hosts everything works except modules/access.t with 31 failures.
>    After reading that code, I see that
>    a) mod_access_compat makes a reverse lookup if it thinks the deny/allow is a host name and lookups are allowed.
>    b) HostnameLookups is 'Off' in the test config
>    c) /etc/hosts does not enable reverse lookups for the resolver (at least not on ubuntu/osx)
>
> So, it seems to me you all are testing with "127.0.0.1" or enable lookups and have a name from a DNS zone with proper reverse. Everything else should not work.

Testing with localhost here.

No precise idea here about your problem, but

- make sure you don't have any proxy shell environment variables set 
(http_proxy, https_proxy or similar)

- we might get an idea if we can see your /etc/hosts and 
/etc/resolv.conf (cleared from private info).

> //Stefan (needs a drink)

Taming the beast makes one thirsty.

Rainer


Re: test base line

Posted by Stefan Eissing <st...@greenbytes.de>.
Ok, narrowing it down, but not done yet. Problem is as follows:

1 with 'localhost' as servername, tests were failing with "waiting for server to warm up", although server was reachable as http://localhost:8529/
  strange.
2 with 'test.example.org' as servername and entry in /etc/hosts everything works except modules/access.t with 31 failures.
  After reading that code, I see that
  a) mod_access_compat makes a reverse lookup if it thinks the deny/allow is a host name and lookups are allowed. 
  b) HostnameLookups is 'Off' in the test config
  c) /etc/hosts does not enable reverse lookups for the resolver (at least not on ubuntu/osx)

So, it seems to me you all are testing with "127.0.0.1" or enable lookups and have a name from a DNS zone with proper reverse. Everything else should not work. 

//Stefan (needs a drink)

PS. And yes, 2.4.16 seems to work on Ubuntu 14.04 LTS.

> Am 13.07.2015 um 16:02 schrieb Jim Jagielski <ji...@jaguNET.com>:
> 
> If you are using something like a hostname of foo.localdomain (or
> whatever), make sure that /etc/hosts has that resolving to 127.0.0.1
> 
>> On Jul 13, 2015, at 9:50 AM, Stefan Eissing <st...@eissing.org> wrote:
>> 
>> Rainer, 
>> 
>> many thanks for the detailed description. I installed all the perl modules you listed, cleaned the test conf, rebuild the httpd with reallyreallyall modules and now the tests are running *almost* fine.
>> 
>> I get 31 failures in modules/access.t and, frankly, cannot figure out what is wrong on my system. This seem basic grant/deny tests.
>> Test Summary Report
>> -------------------
>> t/modules/access.t                (Wstat: 0 Tests: 408 Failed: 31)
>> Failed tests:  4, 20-21, 24, 26, 28, 30, 38, 55, 72, 89
>>               106-107, 123-124, 141, 154, 168, 170, 175
>>               192, 209, 226, 277, 290, 304, 306, 311
>>               328, 345, 362
>> Files=110, Tests=4312, 72 wallclock secs ( 1.69 usr  0.17 sys + 32.46 cusr  8.66 csys = 42.98 CPU)
>> Result: FAIL
>> Failed 1/110 test programs. 31/4312 subtests failed.
>> 
>> Since it matches the remote ip/host, it must be something in my name resolution, I assume? Does that ring a bell with anyone?
>> 
>> PS. Btw. to eventually be helpful, I switched testing from trunk to the 2.4.16. Same access errors, but everything else runs. (Ubuntu 14.04 LTS x86_64)
>> 
>> 
>>> Am 11.07.2015 um 12:44 schrieb Rainer Jung <ra...@kippdata.de>:
>>> 
>>> Hi Stefan,
>>> 
>>> Am 09.07.2015 um 13:46 schrieb Stefan Eissing:
>>>> I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.
>>>> 
>>>> I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as
>>>> "t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
>>>> Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
>>>> "
>>>> My perl is the default Ubuntu 14.04 perl 5.18.
>>>> 
>>>> Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...
>>> 
>>> I typically use the default config from fresh build I do with configure flags --enable-modules=reallyall and --enable-load-all-modules.
>>> 
>>> I don't get failures as described by you above. I typically run the perl framework with perl plus locally installed modules. To instal modules as a normal user separate from the system installed perl I use local::lib. The stuff I add is Bundle::ApacheTest and recent versions of Test::Harness, Crypt::SSLeay, Net:SSLeay, IO::Socket::SSL, LWP::Protocol::https, HTTP::DAV (plus whatever cpan automatically adds as further dependencies). The list probably could be shortened, but that's the cruft I accumulated over time. When building the HTTPS/SSL parts one must be careful to use the same OpenSSL version that one uses to build the web server. Sometimes this is a bit tricky.
>>> 
>>> The failure in line 19 you describe happens at the end of the following snippet:
>>> 
>>> my $sock = Apache::TestRequest::vhost_socket();
>>> ok $sock && $sock->connected;
>>> 
>>> my $req = "GET @"."localhost/foobar.html HTTP/1.1\r\n".
>>> "Host: " . Apache::TestRequest::hostport() . "\r\n".
>>>  "\r\n";
>>> 
>>> ok $sock->print($req);
>>> 
>>> So it seems $sock is not defined. And indeed the failure in line 13 is the ok check in the second code line above. So the test could not connect to the vhost.
>>> 
>>> Using t/TEST (try help or -help or -h to see the options) you can also just start the web server configured for the tests without immediately running them. You can then try to connect yourself.
>>> 
>>> You can also edit LogLevel in Apache-Test/lib/Apache/TestConfig.pm and increase it before the perl Makefile.PL and the t/TEST to get more log output.
>>> 
>>> Not likely but maybe your system openssl is used by perl and can't connect to a vhost powered by some other OpenSSL that you build your web server against?
>>> 
>>> The vhost_socket() used by the test is defined in lib/Apache/TestRequest.pm as:
>>> 
>>> sub vhost_socket {
>>>  my $module = shift;
>>>  local $Apache::TestRequest::Module = $module if $module;
>>> 
>>>  my $hostport = hostport(Apache::Test::config());
>>> 
>>>  my($host, $port) = split ':', $hostport;
>>>  my(%args) = (PeerAddr => $host, PeerPort => $port);
>>> 
>>>  if ($module and $module =~ /ssl/) {
>>>      require Net::SSL;
>>>      local $ENV{https_proxy} ||= ""; #else uninitialized value in Net/SSL.pm
>>>      return Net::SSL->new(%args, Timeout => UA_TIMEOUT);
>>>  }
>>>  else {
>>>      require IO::Socket;
>>>      return IO::Socket::INET->new(%args);
>>>  }
>>> }
>>> 
>>> Maybe you can add some debug output to STDOUT there to see to which socket it tries to connect and where it fails.
>>> 
>>> Finally: any locally active pieces of security software intercepting the connect?
>>> 
>>> Regards,
>>> 
>>> Rainer
>> 
> 

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Re: test base line

Posted by Jim Jagielski <ji...@jaguNET.com>.
If you are using something like a hostname of foo.localdomain (or
whatever), make sure that /etc/hosts has that resolving to 127.0.0.1

> On Jul 13, 2015, at 9:50 AM, Stefan Eissing <st...@eissing.org> wrote:
> 
> Rainer, 
> 
> many thanks for the detailed description. I installed all the perl modules you listed, cleaned the test conf, rebuild the httpd with reallyreallyall modules and now the tests are running *almost* fine.
> 
> I get 31 failures in modules/access.t and, frankly, cannot figure out what is wrong on my system. This seem basic grant/deny tests.
> Test Summary Report
> -------------------
> t/modules/access.t                (Wstat: 0 Tests: 408 Failed: 31)
>  Failed tests:  4, 20-21, 24, 26, 28, 30, 38, 55, 72, 89
>                106-107, 123-124, 141, 154, 168, 170, 175
>                192, 209, 226, 277, 290, 304, 306, 311
>                328, 345, 362
> Files=110, Tests=4312, 72 wallclock secs ( 1.69 usr  0.17 sys + 32.46 cusr  8.66 csys = 42.98 CPU)
> Result: FAIL
> Failed 1/110 test programs. 31/4312 subtests failed.
> 
> Since it matches the remote ip/host, it must be something in my name resolution, I assume? Does that ring a bell with anyone?
> 
> PS. Btw. to eventually be helpful, I switched testing from trunk to the 2.4.16. Same access errors, but everything else runs. (Ubuntu 14.04 LTS x86_64)
> 
> 
>> Am 11.07.2015 um 12:44 schrieb Rainer Jung <ra...@kippdata.de>:
>> 
>> Hi Stefan,
>> 
>> Am 09.07.2015 um 13:46 schrieb Stefan Eissing:
>>> I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.
>>> 
>>> I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as
>>> "t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
>>> Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
>>> "
>>> My perl is the default Ubuntu 14.04 perl 5.18.
>>> 
>>> Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...
>> 
>> I typically use the default config from fresh build I do with configure flags --enable-modules=reallyall and --enable-load-all-modules.
>> 
>> I don't get failures as described by you above. I typically run the perl framework with perl plus locally installed modules. To instal modules as a normal user separate from the system installed perl I use local::lib. The stuff I add is Bundle::ApacheTest and recent versions of Test::Harness, Crypt::SSLeay, Net:SSLeay, IO::Socket::SSL, LWP::Protocol::https, HTTP::DAV (plus whatever cpan automatically adds as further dependencies). The list probably could be shortened, but that's the cruft I accumulated over time. When building the HTTPS/SSL parts one must be careful to use the same OpenSSL version that one uses to build the web server. Sometimes this is a bit tricky.
>> 
>> The failure in line 19 you describe happens at the end of the following snippet:
>> 
>> my $sock = Apache::TestRequest::vhost_socket();
>> ok $sock && $sock->connected;
>> 
>> my $req = "GET @"."localhost/foobar.html HTTP/1.1\r\n".
>>  "Host: " . Apache::TestRequest::hostport() . "\r\n".
>>   "\r\n";
>> 
>> ok $sock->print($req);
>> 
>> So it seems $sock is not defined. And indeed the failure in line 13 is the ok check in the second code line above. So the test could not connect to the vhost.
>> 
>> Using t/TEST (try help or -help or -h to see the options) you can also just start the web server configured for the tests without immediately running them. You can then try to connect yourself.
>> 
>> You can also edit LogLevel in Apache-Test/lib/Apache/TestConfig.pm and increase it before the perl Makefile.PL and the t/TEST to get more log output.
>> 
>> Not likely but maybe your system openssl is used by perl and can't connect to a vhost powered by some other OpenSSL that you build your web server against?
>> 
>> The vhost_socket() used by the test is defined in lib/Apache/TestRequest.pm as:
>> 
>> sub vhost_socket {
>>   my $module = shift;
>>   local $Apache::TestRequest::Module = $module if $module;
>> 
>>   my $hostport = hostport(Apache::Test::config());
>> 
>>   my($host, $port) = split ':', $hostport;
>>   my(%args) = (PeerAddr => $host, PeerPort => $port);
>> 
>>   if ($module and $module =~ /ssl/) {
>>       require Net::SSL;
>>       local $ENV{https_proxy} ||= ""; #else uninitialized value in Net/SSL.pm
>>       return Net::SSL->new(%args, Timeout => UA_TIMEOUT);
>>   }
>>   else {
>>       require IO::Socket;
>>       return IO::Socket::INET->new(%args);
>>   }
>> }
>> 
>> Maybe you can add some debug output to STDOUT there to see to which socket it tries to connect and where it fails.
>> 
>> Finally: any locally active pieces of security software intercepting the connect?
>> 
>> Regards,
>> 
>> Rainer
> 


Re: test base line

Posted by Eric Covener <co...@gmail.com>.
On Mon, Jul 13, 2015 at 9:50 AM, Stefan Eissing <st...@eissing.org> wrote:
> I get 31 failures in modules/access.t and, frankly, cannot figure out what is wrong on my system. This seem basic grant/deny tests.
> Test Summary Report
> -------------------
> t/modules/access.t                (Wstat: 0 Tests: 408 Failed: 31)
>   Failed tests:  4, 20-21, 24, 26, 28, 30, 38, 55, 72, 89
>                 106-107, 123-124, 141, 154, 168, 170, 175
>                 192, 209, 226, 277, 290, 304, 306, 311
>                 328, 345, 362
> Files=110, Tests=4312, 72 wallclock secs ( 1.69 usr  0.17 sys + 32.46 cusr  8.66 csys = 42.98 CPU)
> Result: FAIL
> Failed 1/110 test programs. 31/4312 subtests failed.
>
> Since it matches the remote ip/host, it must be something in my name resolution, I assume? Does that ring a bell with anyone?
>
> PS. Btw. to eventually be helpful, I switched testing from trunk to the 2.4.16. Same access errors, but everything else runs. (Ubuntu 14.04 LTS x86_64)


Maybe some non world readable/executable directory above docroot like $HOME?

Re: test base line

Posted by Stefan Eissing <st...@eissing.org>.
Rainer, 

many thanks for the detailed description. I installed all the perl modules you listed, cleaned the test conf, rebuild the httpd with reallyreallyall modules and now the tests are running *almost* fine.

I get 31 failures in modules/access.t and, frankly, cannot figure out what is wrong on my system. This seem basic grant/deny tests.
Test Summary Report
-------------------
t/modules/access.t                (Wstat: 0 Tests: 408 Failed: 31)
  Failed tests:  4, 20-21, 24, 26, 28, 30, 38, 55, 72, 89
                106-107, 123-124, 141, 154, 168, 170, 175
                192, 209, 226, 277, 290, 304, 306, 311
                328, 345, 362
Files=110, Tests=4312, 72 wallclock secs ( 1.69 usr  0.17 sys + 32.46 cusr  8.66 csys = 42.98 CPU)
Result: FAIL
Failed 1/110 test programs. 31/4312 subtests failed.

Since it matches the remote ip/host, it must be something in my name resolution, I assume? Does that ring a bell with anyone?

PS. Btw. to eventually be helpful, I switched testing from trunk to the 2.4.16. Same access errors, but everything else runs. (Ubuntu 14.04 LTS x86_64)


> Am 11.07.2015 um 12:44 schrieb Rainer Jung <ra...@kippdata.de>:
> 
> Hi Stefan,
> 
> Am 09.07.2015 um 13:46 schrieb Stefan Eissing:
>> I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.
>> 
>> I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as
>> "t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
>> Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
>> "
>> My perl is the default Ubuntu 14.04 perl 5.18.
>> 
>> Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...
> 
> I typically use the default config from fresh build I do with configure flags --enable-modules=reallyall and --enable-load-all-modules.
> 
> I don't get failures as described by you above. I typically run the perl framework with perl plus locally installed modules. To instal modules as a normal user separate from the system installed perl I use local::lib. The stuff I add is Bundle::ApacheTest and recent versions of Test::Harness, Crypt::SSLeay, Net:SSLeay, IO::Socket::SSL, LWP::Protocol::https, HTTP::DAV (plus whatever cpan automatically adds as further dependencies). The list probably could be shortened, but that's the cruft I accumulated over time. When building the HTTPS/SSL parts one must be careful to use the same OpenSSL version that one uses to build the web server. Sometimes this is a bit tricky.
> 
> The failure in line 19 you describe happens at the end of the following snippet:
> 
> my $sock = Apache::TestRequest::vhost_socket();
> ok $sock && $sock->connected;
> 
> my $req = "GET @"."localhost/foobar.html HTTP/1.1\r\n".
>   "Host: " . Apache::TestRequest::hostport() . "\r\n".
>    "\r\n";
> 
> ok $sock->print($req);
> 
> So it seems $sock is not defined. And indeed the failure in line 13 is the ok check in the second code line above. So the test could not connect to the vhost.
> 
> Using t/TEST (try help or -help or -h to see the options) you can also just start the web server configured for the tests without immediately running them. You can then try to connect yourself.
> 
> You can also edit LogLevel in Apache-Test/lib/Apache/TestConfig.pm and increase it before the perl Makefile.PL and the t/TEST to get more log output.
> 
> Not likely but maybe your system openssl is used by perl and can't connect to a vhost powered by some other OpenSSL that you build your web server against?
> 
> The vhost_socket() used by the test is defined in lib/Apache/TestRequest.pm as:
> 
> sub vhost_socket {
>    my $module = shift;
>    local $Apache::TestRequest::Module = $module if $module;
> 
>    my $hostport = hostport(Apache::Test::config());
> 
>    my($host, $port) = split ':', $hostport;
>    my(%args) = (PeerAddr => $host, PeerPort => $port);
> 
>    if ($module and $module =~ /ssl/) {
>        require Net::SSL;
>        local $ENV{https_proxy} ||= ""; #else uninitialized value in Net/SSL.pm
>        return Net::SSL->new(%args, Timeout => UA_TIMEOUT);
>    }
>    else {
>        require IO::Socket;
>        return IO::Socket::INET->new(%args);
>    }
> }
> 
> Maybe you can add some debug output to STDOUT there to see to which socket it tries to connect and where it fails.
> 
> Finally: any locally active pieces of security software intercepting the connect?
> 
> Regards,
> 
> Rainer


Re: test base line

Posted by Rainer Jung <ra...@kippdata.de>.
Hi Stefan,

Am 09.07.2015 um 13:46 schrieb Stefan Eissing:
> I need some help with establishing a test baseline. I checked out the test framework from  https://svn.apache.org/repos/asf/httpd/test/framework/trunk, followed the README and ran the tests against a freshly installed 2.4.x in /opt/httpd/2.4-plain. It did PASS with the default httpd.conf, but many tests were skipped due to modules missing.
>
> I tried enable some more modules like mod_ssl or mod_rewrite and all of these attempts led to test failures and perl errors such as
> "t/security/CVE-2011-3368-rewrite.t .. 1/3 # Failed test 1 in t/security/CVE-2011-3368-rewrite.t at line 13
> Can't call method "print" on an undefined value at t/security/CVE-2011-3368-rewrite.t line 19.
> "
> My perl is the default Ubuntu 14.04 perl 5.18.
>
> Is this a failure on my part or is the system supposed to operate like this? I am a bit confused...

I typically use the default config from fresh build I do with configure 
flags --enable-modules=reallyall and --enable-load-all-modules.

I don't get failures as described by you above. I typically run the perl 
framework with perl plus locally installed modules. To instal modules as 
a normal user separate from the system installed perl I use local::lib. 
The stuff I add is Bundle::ApacheTest and recent versions of 
Test::Harness, Crypt::SSLeay, Net:SSLeay, IO::Socket::SSL, 
LWP::Protocol::https, HTTP::DAV (plus whatever cpan automatically adds 
as further dependencies). The list probably could be shortened, but 
that's the cruft I accumulated over time. When building the HTTPS/SSL 
parts one must be careful to use the same OpenSSL version that one uses 
to build the web server. Sometimes this is a bit tricky.

The failure in line 19 you describe happens at the end of the following 
snippet:

my $sock = Apache::TestRequest::vhost_socket();
ok $sock && $sock->connected;

my $req = "GET @"."localhost/foobar.html HTTP/1.1\r\n".
    "Host: " . Apache::TestRequest::hostport() . "\r\n".
     "\r\n";

ok $sock->print($req);

So it seems $sock is not defined. And indeed the failure in line 13 is 
the ok check in the second code line above. So the test could not 
connect to the vhost.

Using t/TEST (try help or -help or -h to see the options) you can also 
just start the web server configured for the tests without immediately 
running them. You can then try to connect yourself.

You can also edit LogLevel in Apache-Test/lib/Apache/TestConfig.pm and 
increase it before the perl Makefile.PL and the t/TEST to get more log 
output.

Not likely but maybe your system openssl is used by perl and can't 
connect to a vhost powered by some other OpenSSL that you build your web 
server against?

The vhost_socket() used by the test is defined in 
lib/Apache/TestRequest.pm as:

sub vhost_socket {
     my $module = shift;
     local $Apache::TestRequest::Module = $module if $module;

     my $hostport = hostport(Apache::Test::config());

     my($host, $port) = split ':', $hostport;
     my(%args) = (PeerAddr => $host, PeerPort => $port);

     if ($module and $module =~ /ssl/) {
         require Net::SSL;
         local $ENV{https_proxy} ||= ""; #else uninitialized value in 
Net/SSL.pm
         return Net::SSL->new(%args, Timeout => UA_TIMEOUT);
     }
     else {
         require IO::Socket;
         return IO::Socket::INET->new(%args);
     }
}

Maybe you can add some debug output to STDOUT there to see to which 
socket it tries to connect and where it fails.

Finally: any locally active pieces of security software intercepting the 
connect?

Regards,

Rainer