[jira] [Commented] (DRILL-5318) Create a sub-operator test framework

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954361#comment-15954361 ] 

ASF GitHub Bot commented on DRILL-5318:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/788#discussion_r109538974
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/ConfigBuilder.java ---
    @@ -0,0 +1,130 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + ******************************************************************************/
    +package org.apache.drill.test;
    +
    +import java.util.Properties;
    +import java.util.Map.Entry;
    +
    +import org.apache.drill.common.config.DrillConfig;
    +
    +/**
    + * Builds a {@link DrillConfig} for use in tests. Use this when a config
    + * is needed by itself, separate from an embedded Drillbit.
    + */
    +public class ConfigBuilder {
    +
    +  protected String configResource;
    +  protected Properties configProps;
    +
    +  /**
    +   * Use the given configuration properties as overrides.
    +   * @param configProps a collection of config properties
    +   * @return this builder
    +   * @see {@link #configProperty(String, Object)}
    +   */
    +
    +  public ConfigBuilder configProps(Properties configProps) {
    +    if (hasResource()) {
    +      // Drill provides no constructor for this use case.
    +      throw new IllegalArgumentException( "Cannot provide both a config resource and config properties.");
    +    }
    +    if (this.configProps == null) {
    +      this.configProps = configProps;
    +    } else {
    +      this.configProps.putAll(configProps);
    +    }
    +    return this;
    +  }
    +
    +  /**
    +   * Use the given configuration file, stored as a resource, to initialize
    +   * the Drill config. Note that the resource file should have the two
    +   * following settings to work as a config for an embedded Drillbit:
    +   * <pre><code>
    +   * drill.exec.sys.store.provider.local.write : false,
    +   * drill.exec.http.enabled : false
    +   * </code></pre>
    +   * It may be more convenient to add your settings to the default
    +   * config settings with {@link #configProperty(String, Object)}.
    +   * @param configResource path to the file that contains the
    +   * config file to be read
    +   * @return this builder
    +   * @see {@link #configProperty(String, Object)}
    +   */
    +
    +  public ConfigBuilder resource(String configResource) {
    +
    +    if (configProps != null) {
    +      // Drill provides no constructor for this use case.
    +      throw new IllegalArgumentException( "Cannot provide both a config resource and config properties.");
    +    }
    +
    +    // TypeSafe gets unhappy about a leading slash, but other functions
    +    // require it. Silently discard the leading slash if given to
    +    // preserve the test writer's sanity.
    +
    +    this.configResource = ClusterFixture.trimSlash(configResource);
    +    return this;
    +  }
    +
    +  /**
    +   * Add an additional boot-time property for the embedded Drillbit.
    +   * @param key config property name
    +   * @param value property value
    +   * @return this builder
    +   */
    +
    +  public ConfigBuilder put(String key, Object value) {
    +    if (hasResource()) {
    +      // Drill provides no constructor for this use case.
    +      throw new IllegalArgumentException( "Cannot provide both a config resource and config properties.");
    +    }
    +    if (configProps == null) {
    +      configProps = new Properties();
    +    }
    +    configProps.put(key, value.toString());
    +    return this;
    +  }
    +
    +  public DrillConfig build() {
    +
    +    // Create a config
    +    // Because of the way DrillConfig works, we can set the ZK
    +    // connection string only if a property set is provided.
    +
    +    if (hasResource()) {
    +      return DrillConfig.create(configResource);
    +    } else if (configProps != null) {
    +      return DrillConfig.create(configProperties());
    +    } else {
    +      return DrillConfig.create();
    +    }
    +  }
    +
    +  private Properties configProperties() {
    --- End diff --
    
    Here is what should be happening, let me know if you see something different.
    
    For convenience, the config builder lets you specify property values as typed values:
    ```
    builder.put("some.boolean", true);
    builder.put("some.int", 10);
    builder.put("some.string", "foo");
    ```
    
    The TypeSafe config system wants properties, but they must all be in the form of strings. For the above, the values must be "true", "10" and "foo".
    
    This method translates the typed values into strings of the form needed by TypeSafe. Notice the loop over the "configProps", conversion to string, and the insertion into a newly created "effectiveProps."


> Create a sub-operator test framework
> ------------------------------------
>
>                 Key: DRILL-5318
>                 URL: https://issues.apache.org/jira/browse/DRILL-5318
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Tools, Build & Test
>    Affects Versions: 1.11.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>             Fix For: 1.11.0
>
>         Attachments: Sub-OperatorTestFramework.pdf
>
>
> Drill provides two unit test frameworks for whole-server, SQL-based testing: the original {{BaseTestQuery}} and the newer {{ClusterFixture}}. Both use the {{TestBuilder}} mechanism to build system-level functional tests that run queries and check results.
> Jason provided an operator-level test framework based, in part on mocks: 
> As Drill operators become more complex, we have a crying need for true unit-level tests at a level below the whole system and below operators. That is, we need to test the individual pieces that, together, form the operator.
> This umbrella ticket includes a number of tasks needed to create the sub-operator framework. Our intention is that, over time, as we find the need to revisit existing operators, or create new ones, we can employ the sub-operator test framework to exercise code at a finer granularity than is possible prior to this framework.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)