[jira] [Commented] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050675#comment-17050675 ] 

Duo Zhang commented on HBASE-23779:
-----------------------------------

I think this is the root cause of the recently mess? 0.5C seems a bit aggresive? And I think we should also take care of memory? if we run UTs on a 32 core 64GB memory machine, we will have 16 processes and each consume 4GB of memory(we even need more?), it will easy lead to OOM and cause all UTs to fail randomly...

> Up the default fork count to make builds complete faster; make count relative to CPU count
> ------------------------------------------------------------------------------------------
>
>                 Key: HBASE-23779
>                 URL: https://issues.apache.org/jira/browse/HBASE-23779
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>            Reporter: Michael Stack
>            Assignee: Michael Stack
>            Priority: Major
>             Fix For: 3.0.0, 2.3.0
>
>         Attachments: addendum2.patch, test_yetus_934.0.patch
>
>
> Tests take a long time. Our fork count running all tests are conservative -- 1 (small) for first part and 5 for second part (medium and large). Rather than hardcoding we should set the fork count to be relative to machine size. Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going by my random survey. The load reported on a few seems low though this not representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look see.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)