You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Artem Kuchin <ma...@itlegion.ru> on 2008/06/19 13:51:55 UTC
[users@httpd] Weird Ip in access log, need help to understand
I recently upgraded to 2.2 and now i see thing which i have never seen
mefore in
the access log:
74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET
/Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0 (compatible;
Yahoo! Slurp; http://he
lp.yahoo.com/help/us/ysearch/slurp)"
What is "74.6.29.159, 74.6.8.118" ?
Some records how even three IP addesses! Some have the word "unknown"
then comma ip address.
i have
HostnameLookups Off
in the config file.
What is this? How is it possible? What does it mean?
--
Artem
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Weird Ip in access log, need help to understand
Posted by André Warnier <aw...@ice-sa.com>.
Artem Kuchin wrote:
>
>
> André Warnier пишет:
>>
>>
>> Artem Kuchin wrote:
>>> I recently upgraded to 2.2 and now i see thing which i have never
>>> seen mefore in
>>> the access log:
>>>
>>> 74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET
>>> /Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0
>>> (compatible; Yahoo! Slurp; http://he
>>> lp.yahoo.com/help/us/ysearch/slurp)"
>>
>> Following the link to :
>> http://help.yahoo.com//help/us/ysearch/slurp
>> may help understand
>>
>
> no, it does not. The problem is not with slurp and is not related to any
> particular client. It is not even
> a problem in a common meaning. The question is WHY in this record are 2
> (TWO) ip address in
> the first field of the record: 74.6.29.159, 74.6.8.118
> How it is possible at all? One request comes only from one ip. Where
> apache takes these
> addresses?
>
Well ok.
Now both IP's are indeed from yahoo's crawlers (of which there are
probably many hundreds) :
dent:~# nslookup 74.6.29.159
Non-authoritative answer:
159.29.6.74.in-addr.arpa name = lj512625.crawl.yahoo.net.
dent:~# nslookup 74.6.8.118
Non-authoritative answer:
118.8.6.74.in-addr.arpa name = llf520026.crawl.yahoo.net.
What I would guess is that Apache somehow regroups identical and
simultaneous requests into a single log line. And I would guess that
Yahoo crawlers work "in teams", so that it is not surprising that
several requests would arrive simultaneously to your server.
I'm just guessing here.
But it's interesting anyway. I have not really looked specifically, but
I don't recall seeing such lines on any of our servers.
A question : is the URL in those accesses, a real URL that exists on
your site ? I see a 200 OK return code, but some sites return a page
even for an invalid URL.
If it is not, then there is one item on the Yahoo page that may explain
why there are such quasi-simultaneous requests. I think it's called
"why is yahoo trying to get strange urls from my site ?"
André
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Weird Ip in access log, need help to understand
Posted by André Warnier <aw...@ice-sa.com>.
I just saw the following in the documentation for mod_proxy :
quote
Be careful when using these headers on the origin server, since they
will contain more than one (comma-separated) value if the original
request already contained one of these headers. For example, you can use
%{X-Forwarded-For}i in the log format string of the origin server to log
the original clients IP address, but you may get more than one address
if the request passes through several proxies.
unquote
Maybe this has something to do with the issue you mention ?
I mean that maybe yahoo requests go through several proxies, and what
you are seeing in your loogs, is the result of a collection of various
proxy addresses, decoded by your logging format.
André
Artem Kuchin wrote:
>
>
> André Warnier пишет:
>>
>>
>> Artem Kuchin wrote:
>>> I recently upgraded to 2.2 and now i see thing which i have never
>>> seen mefore in
>>> the access log:
>>>
>>> 74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET
>>> /Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0
>>> (compatible; Yahoo! Slurp; http://he
>>> lp.yahoo.com/help/us/ysearch/slurp)"
>>
>> Following the link to :
>> http://help.yahoo.com//help/us/ysearch/slurp
>> may help understand
>>
>
> no, it does not. The problem is not with slurp and is not related to any
> particular client. It is not even
> a problem in a common meaning. The question is WHY in this record are 2
> (TWO) ip address in
> the first field of the record: 74.6.29.159, 74.6.8.118
> How it is possible at all? One request comes only from one ip. Where
> apache takes these
> addresses?
>
> --
> Regards,
> Artem
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Weird Ip in access log, need help to understand
Posted by Artem Kuchin <ma...@itlegion.ru>.
André Warnier пишет:
>
>
> Artem Kuchin wrote:
>> I recently upgraded to 2.2 and now i see thing which i have never
>> seen mefore in
>> the access log:
>>
>> 74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET
>> /Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0
>> (compatible; Yahoo! Slurp; http://he
>> lp.yahoo.com/help/us/ysearch/slurp)"
>
> Following the link to :
> http://help.yahoo.com//help/us/ysearch/slurp
> may help understand
>
no, it does not. The problem is not with slurp and is not related to any
particular client. It is not even
a problem in a common meaning. The question is WHY in this record are 2
(TWO) ip address in
the first field of the record: 74.6.29.159, 74.6.8.118
How it is possible at all? One request comes only from one ip. Where
apache takes these
addresses?
--
Regards,
Artem
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Weird Ip in access log, need help to understand
Posted by André Warnier <aw...@ice-sa.com>.
Artem Kuchin wrote:
> I recently upgraded to 2.2 and now i see thing which i have never seen
> mefore in
> the access log:
>
> 74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET
> /Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0 (compatible;
> Yahoo! Slurp; http://he
> lp.yahoo.com/help/us/ysearch/slurp)"
Following the link to :
http://help.yahoo.com//help/us/ysearch/slurp
may help understand
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org