[jira] [Created] (HBASE-25081) Up the container nproc uplimit to 30000

["Istvan Toth (Jira)" <ji...@apache.org>]

Istvan Toth created HBASE-25081:
-----------------------------------

             Summary: Up the container nproc uplimit to 30000
                 Key: HBASE-25081
                 URL: https://issues.apache.org/jira/browse/HBASE-25081
             Project: HBase
          Issue Type: Bug
          Components: test
            Reporter: Istvan Toth


We (Apache Phoenix team) have recently switched our precommit tests to Dockerized Yetus (mostly adopted from the solution in Hbase)

We see 
java.lang.OutOfMemoryError: unable to create new native thread
errors , while Yetus shows
|Max. process+thread count|6833 (vs. ulimit of 12500)|

While I couldn't determine what was the job that we shared the Agent with at the time, statistically it was very likely HBase, and an HBase job probably failed with a similar error.

Some research has thrown up the official Docker docs: 

[https://docs.docker.com/engine/reference/commandline/run/#set-ulimits-in-container-ulimit]

According to which it is not possible to set container level nprocs ulimit with Docker.

All settings apply to the docker Daemon user instead, and the limit is shared between all containers.

Based on this, I think that it makes no sense to set a container (really docker user) nprocs ulimit any lower than the current hard limit of 30000.

I have already set PROC_LIMIT=30000 in the Phoenix Yetus personality, but it is only a half solution until some Docker users set lower values, as the later setting will apply as soon as the container is started.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)