[jira] [Commented] (DRILL-5122) DrillBuf performs very expensive logging if JVM -ea option is set

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15738538#comment-15738538 ] 

Paul Rogers commented on DRILL-5122:
------------------------------------

Easiest possible fix:
{code}
  public static final boolean DEBUG = AssertionUtil.isAssertionsEnabled()
      && Boolean.parseBoolean(System.getProperty(DEBUG_ALLOCATOR, "true"));
{code}

That is, reverse the polarity of the {{DEBUG_ALLOCATOR}} command line option. This is probably safe because the folks that worked on the memory allocator are not currently active on the project, so this won't break any existing debug setups.

> DrillBuf performs very expensive logging if JVM -ea option is set
> -----------------------------------------------------------------
>
>                 Key: DRILL-5122
>                 URL: https://issues.apache.org/jira/browse/DRILL-5122
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components:  Server
>    Affects Versions: 1.8.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>            Priority: Minor
>
> The {{DrillBuf}} class is the foundation of Drill's direct memory model. It has a very sophisticated allocator mechanism behind it. {{DrillBuf}} provides logging to help track down memory leaks and other issues. Unfortunately, the trigger to turn on this option is based on detecting if assertions are enabled in the JVM.
> Assertions are very handy tools: they are debug-only assertions not executed at run time. They are turned on with the "-ea" option to the JVM. But, in production, the assertions don't execute, avoiding production-time costs.
> In Drill, once assertions are enabled, the dominant cost of a query becomes {{DrillBuf}} logging. On example showed that {{DrillBuf}} logging increases query cost by 10x.
> Debugging of {{DrillBuf}} is handy for those few engineers who need to do so. Over time, {{DrillBuf}} has become quite stable so that, now, most developers, most of the time, do not work on {{DrillBuf}}.
> One wonders: Drill uses Guava preconditions extensively. Is this perhaps due to the high cost of enabling assertions? Since assertions were unavailable, production-time checking via Guava may have become preferred. Assertions are, in fact, better because they incur no overhead in production, while Guava preconditions always execute, even in production. (Here we assume that the code errors that both check will find errors during development so that such errors do not sneak through into production.)
> The code does have a way to turn off debugging, but only if a system property is set on every run. Since Drill has hundreds of unit tests that people might run, this requires creating custom run configurations for each.
> This enhancement proposes to modify {{DrillBuf}} to enable logging only when a) assertions are enabled (so we know it is a debug environment) and b) when a boot-time configuration property is set (so that logging can be turned off for most developers most of the time.) Optionally, b) can be a system property that _enables_ (not disables) {{DrillBuf}} logging.
> The code in question:
> {code}
> class BaseAllocator ... {
>   public static final boolean DEBUG = AssertionUtil.isAssertionsEnabled()
>       || Boolean.parseBoolean(System.getProperty(DEBUG_ALLOCATOR, "false"));
> ...
> class DrillBuf ... {
>   public boolean release(int decrement) {
>     ...
>     if (BaseAllocator.DEBUG) {
>       historicalLog.recordEvent("release(%d). original value: %d", decrement, refCnt + decrement);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)