You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Sebastian Nagel (JIRA)" <ji...@apache.org> on 2017/09/22 11:12:00 UTC

[jira] [Closed] (NUTCH-2212) Decrease memory consumption by tuning stack size

     [ https://issues.apache.org/jira/browse/NUTCH-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebastian Nagel closed NUTCH-2212.
----------------------------------

> Decrease memory consumption by tuning stack size
> ------------------------------------------------
>
>                 Key: NUTCH-2212
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2212
>             Project: Nutch
>          Issue Type: Task
>    Affects Versions: 1.11
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>            Priority: Minor
>             Fix For: 1.14
>
>
> In today's environments it is common to see a default stack size (-Xss) of 1 MB. This is ridiculous for a fetcher running many fetcher.threads.fetch. The actual number of threads is much higher due to parsing and running it on YARN. 
> We can decrease stack usage by 75 %, 1 MB to a safe 256 kB. YARN will run out of stack size if we set it to 128 kB.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)