You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sebastian Nagel (Jira)" <ji...@apache.org> on 2019/10/15 12:04:00 UTC
[jira] [Resolved] (NUTCH-2511) SitemapProcessor limited by
http.content.limit
[ https://issues.apache.org/jira/browse/NUTCH-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Nagel resolved NUTCH-2511.
------------------------------------
Resolution: Fixed
Thanks, [~yossi]!
> SitemapProcessor limited by http.content.limit
> ----------------------------------------------
>
> Key: NUTCH-2511
> URL: https://issues.apache.org/jira/browse/NUTCH-2511
> Project: Nutch
> Issue Type: Bug
> Components: sitemap
> Affects Versions: 1.14
> Reporter: Yossi Tamari
> Priority: Major
> Fix For: 1.17
>
>
> Because SitemapProcessor uses the HTTP protocol plugin, which limits the size of a response to http.content.limit (64KB by default), it can only handle sitemaps smaller than that size.
> I don't believe that is the intent of the users by setting http.content.limit - they want to limit document size, not sitemap size. The spec specifically says that sitemaps can be up to 50MB.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)