[jira] [Commented] (COUCHDB-1282) Allow special value 0 for os_process_limit

["Randall Leeds (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COUCHDB-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105059#comment-13105059 ] 

Randall Leeds commented on COUCHDB-1282:
----------------------------------------

Background:
This work stemmed from some experiments I conducted with refactoring the bulk update/validation code in couch_db to validate in parallel. I found that when I did not restrict how many os processes it used, and therefore used the full 25 default, performance was slightly worse than when using only 2 or 4. However, almost all of this performance can be won by breaking the request at the client into pieces and performing parallel bulk updates there. Adam also correctly points out that view indexing ties up os processes for the entire duration of a view index update and restricted this to such small numbers could potentially lead to all the query server processes getting tied up.

Closing this for now. Ping me if you want to pick up a conversation about query server performance.

> Allow special value 0 for os_process_limit
> ------------------------------------------
>
>                 Key: COUCHDB-1282
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1282
>             Project: CouchDB
>          Issue Type: Improvement
>            Reporter: Randall Leeds
>            Assignee: Randall Leeds
>            Priority: Trivial
>         Attachments: 0001-Smart-os_proc_limit-default-COUCHDB-1282.patch
>
>
> It's unlikely to be beneficial to run more os processes than there are logical processors in a system. It creates extra scheduling overhead if they are all in use. Currently default.ini specifies 25, though the code specifies 10 if the option is missing, and I expect both are too high for many real world deployments.
> I propose the value 0 be used in default.ini and that it take special meaning: use the number of logical processors erlang detects.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira