Checking a webserver

[André Warnier <aw...@ice-sa.com>]

Hi.

I am looking for a debugging tool that would be able to repeatedly issue 
HTTP requests to one or more URLs, and log any errors.  Preferably in 
Perl, and it must run on a Windows workstation.

So something like a wget or curl, but with some smart command-line 
parameters allowing it to run in a continuous loop, with an adjustable 
request-issuing rate. Since I know exactly what kind of pages I should 
receive in return, I might want to modify this tool (if it is in Perl), 
to add some HTML parsing looking for specific information on the 
response pages.

I could write something myself using LWP of course, but I figure that it 
is most likely that someone already did this.
My main problem is that I don't know how something like that would be 
called or which keywords to look for.

I would appreciate any pointers, even just the keywords to search.
Thanks in advance.

Re: Checking a webserver

[André Warnier <aw...@ice-sa.com>]

Michael Ludwig wrote:
> Torsten Foertsch schrieb:
>> On Wed 28 Jan 2009, André Warnier wrote:
>>> I am looking for a debugging tool that would be able to repeatedly
>>> issue HTTP requests to one or more URLs, and log any errors.
>>>  Preferably in Perl, and it must run on a Windows workstation.
>>
>> Don't know about windows and its not perl but I use ab (comes with
>> httpd)
> 
> Apache's AB.exe works fine on Windows. But it is for stress testing.
> Is that you want?
> 
> Or is it rather application monitoring you want to do?
> 
It's neither one, or maybe a mixture of both.
I'll try to summarise the case :

We have a customer site (among many with the same setup) experiencing 
some problem : users get "friendly" error messages in IE saying "this 
page cannot be displayed".  IE6 also sometimes adds a status bar message 
saying "Server not found - DNS error" (but not always, it varies).
We cannot change IE6.
The setup is as follows :

Request:
IE6 -> Apache2.2 -> mod_jk1.2.28 -> Tomcat5.5 -> database app.
Response:
IE6 <- Apache2.2 <- mod_jk1.2.28 <- Tomcat5.5 <- database app.

The mod_jk error logs shows messages to the effect that the *response* 
cannot be sent because the client has broken the connection.
Like :

[Mon Jan 19 15:02:52 2009] [6802:4416] [info] 
ajp_process_callback::jk_ajp_common.c (1447): Writing to client aborted 
or client network problems
[Mon Jan 19 15:02:52 2009] [6802:4416] [info] 
ajp_service::jk_ajp_common.c (1846): (ajp13) request failed, because of 
client write error without recovery in send loop attempt=0
(+ many of these)

Normally, that means the user clicked somewhere else, or on cancel 
etc.., but we are pretty sure that it's not the case here (I've seen the 
error myself at the customer site).
So we suspect a problem either with IE, or with the network at the 
client side.

What I want to do, is to run some monitoring program on a client 
station, instead of IE, which should simulate client accesses to the 
same application, and tell me in detail if there is an error, and what 
the error looks like.  Unlike IE.
Then I can, maybe, cross-reference this with the errors seen in the 
mod_jk log and find out if they are the same, or unrelated.
I hope for something that will log if it cannot access the site because 
there is indeed a DNS error finding the server, or because it cannot 
connect to the server, or because the server takes too long, or 
whatever. Or that will tell me that it has no problem at all getting the 
response from the server, which would be a clear sign that it's 
something in IE6.  Which I hope for, because we cannot find errors at 
the app or app-server level so far.

Thanks.

And thanks also for the several responses and tips received so far.

I'm looking at them.  It is not that there are no tools, it's more that 
there are too many.  Preferably, I am looking for something which I can 
ask the customer themselves to install on a workstation and run, because 
that would save me having to go on location again.
I do not have remote access to the client workstations, only to the server.
The workstation I'm thinking of already has perl installed.

I have tried "ab" already locally here, and it is relatively easy to 
install on a Windows workstation (you install Apache for Win32).
One of my problems is that I have a bit of difficulty setting up a local 
case of a network misbehaving from time to time, so I don't really know 
how much debugging information "ab" provides when thrings go wrong.  So 
far, things go desperately right.. ;-)

Re: Checking a webserver

[Michael Ludwig <ml...@as-guides.com>]

Torsten Foertsch schrieb:
> On Wed 28 Jan 2009, André Warnier wrote:
>> I am looking for a debugging tool that would be able to repeatedly
>> issue HTTP requests to one or more URLs, and log any errors.
>>  Preferably in Perl, and it must run on a Windows workstation.
>
> Don't know about windows and its not perl but I use ab (comes with
> httpd)

Apache's AB.exe works fine on Windows. But it is for stress testing.
Is that you want?

Or is it rather application monitoring you want to do?

In case the output is well-formed XML or HTML and you want to use XML
technologies, xmllint or xsltproc in a shell script might be useful.

Michael Ludwig

Re: Checking a webserver

[Torsten Foertsch <to...@gmx.net>]

On Wed 28 Jan 2009, André Warnier wrote:
> I am looking for a debugging tool that would be able to repeatedly
> issue HTTP requests to one or more URLs, and log any errors.
>  Preferably in Perl, and it must run on a Windows workstation.

Don't know about windows and its not perl but I use ab (comes with 
httpd) and flood:

http://httpd.apache.org/test/flood/

Torsten

-- 
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net

Re: Checking a webserver

[André Warnier <aw...@ice-sa.com>]

Cosimo Streppone wrote:
> 
> But after your second message, I thought that maybe you could be
> interested in Fiddler:
> 
>   http://www.fiddlertool.com/Fiddler/help/
> 
I downloaded it and installed it, though I felt a bit unsure about it. 
Is this, or is not, a Microsoft-issued program ?
There seems to be a strange mixture of references and non-references to 
MS on the site and in the product documentation.

It seems to be doing the same kind of things for IE as the various 
plugins for Firefox (LiveHttpHeaders etc..), all-in-one.

However, I don't think that this is "simple enough" for me to be able to 
have the customer download it, install it and run it on a workstation.

But as a developper tool for IE, it looks very nice.

Thanks for the pointer.

Re: Checking a webserver

[Cosimo Streppone <co...@streppone.it>]

In data 28 gennaio 2009 alle ore 11:21:31, André Warnier <aw...@ice-sa.com>  
ha scritto:

> I am looking for a debugging tool that would be able to repeatedly issue  
> HTTP requests to one or more URLs, and log any errors.  Preferably in  
> Perl, and it must run on a Windows workstation.

I was going to say siege:

   http://www.joedog.org/index/siege-home

but I don't think there's a win32 binary.
I evaluated it for some stress testing tools we needed, together
with JMeter and httperf.

But after your second message, I thought that maybe you could be
interested in Fiddler:

   http://www.fiddlertool.com/Fiddler/help/

I think Fiddler is a powerful tool, and you should "waste" some
time learning the various features.

I don't have it at hand now, but I know you can hook your
IE6 test client to it because it acts like a proxy, and
you can record and playback sessions with full logging.

In this way you could let your real users use your application,
record all the session, check eventual errors and then
(maybe, I don't know your app) replay the traffic and see
if the thing happens again.

If you find alternatives, please let me know.

-- 
Cosimo