You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by wangxu <wa...@souchang.com> on 2007/02/08 15:41:03 UTC
why did nutch0.8.1 fetch empty content from certain sites?
when I fetched some certain sites,
I got empty content,contentType,but the fetch status was "fetch_success"
and the metadata was sometimes not empty.
how does website configure itself to achieve this?
any methods to avoid this situation?
I used agent-name:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR 1.1.4322)
Re: why did nutch0.8.1 fetch empty content from certain sites?
Posted by Jason Culverhouse <ja...@mischievous.org>.
It could b related to http://issues.apache.org/jira/browse/NUTCH-374
when the property http.content.limit is set to -1
and the data from the server is gzip'ed the content is not decoded
properly.
Jason
On Feb 8, 2007, at 6:45 AM, wangxu wrote:
> wangxu wrote:
>> when I fetched some certain sites,
>> I got empty content,contentType,but the fetch status was
>> "fetch_success" and the metadata was sometimes not empty.
>>
>> how does website configure itself to achieve this?
>> any methods to avoid this situation?
>> I used agent-name:
>> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR
>> 1.1.4322)
>>
>>
> sorry,empty content,parsedtext/parseddata
Re: why did nutch0.8.1 fetch empty content from certain sites?
Posted by wangxu <wa...@souchang.com>.
wangxu wrote:
> when I fetched some certain sites,
> I got empty content,contentType,but the fetch status was
> "fetch_success" and the metadata was sometimes not empty.
>
> how does website configure itself to achieve this?
> any methods to avoid this situation?
> I used agent-name:
> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR
> 1.1.4322)
>
>
sorry,empty content,parsedtext/parseddata