[jira] [Created] (DRILL-5158) BootStrapContext has scan thread size set to large for embedded

["Paul Rogers (JIRA)" <ji...@apache.org>]

Paul Rogers created DRILL-5158:
----------------------------------

             Summary: BootStrapContext has scan thread size set to large for embedded
                 Key: DRILL-5158
                 URL: https://issues.apache.org/jira/browse/DRILL-5158
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.9.0
            Reporter: Paul Rogers
            Assignee: Parth Chandra
            Priority: Trivial


The Parquet async project added a new config setting: the number of scan threads. But, the hard-coded minimum is too large for an embedded Drillbit on a Mac used for unit tests.

In {{BootStrapContext}}:

{code}
  private static final int MIN_SCAN_THREADPOOL_SIZE = 8; // Magic num
  ...
    final int numScanThreads = (int) (config.getDouble(ExecConstants.SCAN_THREADPOOL_SIZE));
    ...
    final int scanThreadPoolSize =
        MIN_SCAN_THREADPOOL_SIZE > numScanThreads ? MIN_SCAN_THREADPOOL_SIZE : numScanThreads;
{code}

Effectively, this code says you can have as small a thread pool as you like as long as it is 8 or greater.

Empirically, a thread pool size of 2 (set by altering the above code) caused the Drillbit to hang. A size of 4 allowed the Drillbit to run.

In theory, a size of 0 should be fine as long as the code does not use Parquet (or use the async reader). A pool size of 2 should work, but be slow (compared to a larger size.)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)