[jira] [Commented] (AMBARI-11383) Enable async audit log and unfair lock by default in HDP

["GAO Rui (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395140#comment-15395140 ] 

GAO Rui commented on AMBARI-11383:
----------------------------------

Thanks for your contribution, [~sumitmohanty] .

Could you share your opinion about why make default value of dfs.namenode.fslock.fair as false?  

In current HDFS source code, the default value is set as true: 
{code}
    boolean fair = conf.getBoolean("dfs.namenode.fslock.fair", true);
    LOG.info("fsLock is fair:" + fair);
    fsLock = new FSNamesystemLock(fair);
{code}

And, according to the following document, for a busy cluster(where both write and read operations concurrency are very high), it's very easy to cause the problem of writers can never get the lock. Because, once one reader acquire the read lock and mounts of reader keep acquiring read lock, the writers wound got the chance to acquire write lock at all.

https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html
{code}
        Non-fair mode (default)
        When constructed as non-fair (the default), the order of entry to the read and write lock is unspecified, subject to reentrancy constraints. A nonfair lock that is continuously contended may indefinitely postpone one or more reader or writer threads, but will normally have higher throughput than a fair lock.
        Fair mode
        When constructed as fair, threads contend for entry using an approximately arrival-order policy. When the currently held lock is released either the longest-waiting single writer thread will be assigned the write lock, or if there is a group of reader threads waiting longer than all waiting writer threads, that group will be assigned the read lock.
        A thread that tries to acquire a fair read lock (non-reentrantly) will block if either the write lock is held, or there is a waiting writer thread. The thread will not acquire the read lock until after the oldest currently waiting writer thread has acquired and released the write lock. Of course, if a waiting writer abandons its wait, leaving one or more reader threads as the longest waiters in the queue with the write lock free, then those readers will be assigned the read lock.
         
        A thread that tries to acquire a fair write lock (non-reentrantly) will block unless both the read lock and write lock are free (which implies there are no waiting threads). (Note that the non-blocking ReentrantReadWriteLock.ReadLock.tryLock() and ReentrantReadWriteLock.WriteLock.tryLock() methods do not honor this fair setting and will acquire the lock if it is possible, regardless of waiting threads.)
{code}

> Enable async audit log and unfair lock by default in HDP
> --------------------------------------------------------
>
>                 Key: AMBARI-11383
>                 URL: https://issues.apache.org/jira/browse/AMBARI-11383
>             Project: Ambari
>          Issue Type: Bug
>          Components: stacks
>    Affects Versions: 2.1.0
>            Reporter: Sumit Mohanty
>            Assignee: Sumit Mohanty
>            Priority: Critical
>             Fix For: 2.1.0
>
>         Attachments: AMBARI-11383.patch
>
>
> Set the corresponding conf properties:
> dfs.namenode.audit.log.async=true 
> dfs.namenode.fslock.fair=false



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)