general/2908: Problem on crawling an url

[Elena Sotelo <es...@ipmultimedia.es>]

>Number:         2908
>Category:       general
>Synopsis:       Problem on crawling an url
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    apache
>State:          open
>Class:          support
>Submitter-Id:   apache
>Arrival-Date:   Wed Aug 26 10:50:01 PDT 1998
>Last-Modified:
>Originator:     esotelo@ipmultimedia.es
>Organization:
apache
>Release:        1.2.*
>Environment:
Unknown

>Description:
I'm trying to see if an url is a broken link. But when I access an Apache Server I get a 403 Forbidden Error from the server.
If I open the url with a browser, I could get well the url.

Via telnet:
telnet 2000kva.com 80
...
GET / HTTP/1.0
[Server Response]
...
403 Forbidden
...

Must I put anything else in the request??

I get the same response in many Apache Servers (vs. 1.2.4 and 1.2.5)

Another url that I get the same result is : www.gasconysons.com

Any suggests???

Thanks in advance
>How-To-Repeat:
Examples:

>Fix:

>Audit-Trail:
>Unformatted:
[In order for any reply to be added to the PR database, ]
[you need to include <ap...@Apache.Org> in the Cc line ]
[and leave the subject line UNCHANGED.  This is not done]
[automatically because of the potential for mail loops. ]
[If you do not include this Cc, your reply may be ig-   ]
[nored unless you are responding to an explicit request ]
[from a developer.                                      ]
[Reply only with text; DO NOT SEND ATTACHMENTS!         ]


>telnet 2000kva.com 80
>GET / HTTP/1.0

>telnet www.gasconysons.com 80
> GET / HTTP/1.0

On Netscape: the response is well!!