[jira] [Created] (HDFS-11900) HEDGED_READ_THREAD_POOL should not be static

["John Zhuge (JIRA)" <ji...@apache.org>]

John Zhuge created HDFS-11900:
---------------------------------

             Summary: HEDGED_READ_THREAD_POOL should not be static
                 Key: HDFS-11900
                 URL: https://issues.apache.org/jira/browse/HDFS-11900
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs-client
    Affects Versions: 2.8.0
            Reporter: John Zhuge


*Non-static* synchronized method initThreadsNumForHedgedReads can't synchronize the access to the *static* class variable HEDGED_READ_THREAD_POOL.
{code}
  private static ThreadPoolExecutor HEDGED_READ_THREAD_POOL;
...
  private synchronized void initThreadsNumForHedgedReads(int num) {
{code}
2 DFS clients may update the same static variable in a race because the lock is on each DFS client object, not on the shared DFSClient class object.

There are 2 possible fixes:
1. "Global thread pool": Change initThreadsNumForHedgedReads to static
2. "Per-client thread pool": Change HEDGED_READ_THREAD_POOL to non-static

From the description for property {{dfs.client.hedged.read.threadpool.size}}:
{quote}
    to a positive number. The threadpool size is how many threads to dedicate
    to the running of these 'hedged', concurrent reads in your client.
{quote}
it seems to indicate the thread pool is per DFS client.

Let's assume we go with #1 "Global thread pool". One DFS client has the property set to 10 in its config, while the other client has the property set to 5 in its config, what is supposed to the size of the global thread pool? 5? 10? Or 15?

The 2nd fix seems more reasonable to me.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org