You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by jyoti aditya <jy...@gmail.com> on 2016/12/06 11:13:11 UTC

page size

Hi team,

I am getting only around half page of data from a web page when i crawl.
Is there any size limit property to be configured?

With Regards
Jyoti Aditya

Re: page size

Posted by Vincent <vi...@openindex.io>.

Hi Jyoti,

There is a property http.content.limit in the conf. Beyond that size the 
content will be truncated.

Cheers,
Vincent

On 06-12-16 12:13, jyoti aditya wrote:
> Hi team,
>
> I am getting only around half page of data from a web page when i crawl.
> Is there any size limit property to be configured?
>
> With Regards
> Jyoti Aditya
>