[jira] [Resolved] (HBASE-25081) Up the container nproc uplimit to 30000

["Michael Stack (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-25081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Stack resolved HBASE-25081.
-----------------------------------
    Fix Version/s: 3.0.0-alpha-1
     Hadoop Flags: Reviewed
     Release Note: Ups the nproc (processes) limit from 12500 to 30000 in yetus (so build container can have new limit).
         Assignee: Istvan Toth
       Resolution: Fixed

> Up the container nproc uplimit to 30000
> ---------------------------------------
>
>                 Key: HBASE-25081
>                 URL: https://issues.apache.org/jira/browse/HBASE-25081
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>            Reporter: Istvan Toth
>            Assignee: Istvan Toth
>            Priority: Major
>             Fix For: 3.0.0-alpha-1
>
>
> We (Apache Phoenix team) have recently switched our precommit tests to Dockerized Yetus (mostly adopted from the solution in Hbase)
> We see 
> java.lang.OutOfMemoryError: unable to create new native thread
> errors , while Yetus shows
> |Max. process+thread count|6833 (vs. ulimit of 12500)|
> While I couldn't determine what was the job that we shared the Agent with at the time, statistically it was very likely HBase, and an HBase job probably failed with a similar error.
> Some research has thrown up the official Docker docs: 
> [https://docs.docker.com/engine/reference/commandline/run/#set-ulimits-in-container-ulimit]
> According to which it is not possible to set container level nprocs ulimit with Docker.
> All settings apply to the docker Daemon user instead, and the limit is shared between all containers.
> Based on this, I think that it makes no sense to set a container (really docker user) nprocs ulimit any lower than the current hard limit of 30000.
> I have already set PROC_LIMIT=30000 in the Phoenix Yetus personality, but it is only a half solution until some Docker users set lower values, as the later setting will apply as soon as the container is started.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)