[GitHub] drill issue #1114: Drill-6104: Added Logfile Reader

[paul-rogers <gi...@git.apache.org>]

Github user paul-rogers commented on the issue:

    https://github.com/apache/drill/pull/1114
  
    See [this example](https://github.com/paul-rogers/drill/tree/regex-plugin/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/regex), and [this test](https://github.com/paul-rogers/drill/blob/regex-plugin/exec/java-exec/src/test/java/org/apache/drill/exec/store/easy/regex/TestRegexReader.java) for examples of one way to address some of the comments made in the code review. That example handles projection, which was mentioned in one of the review comments.
    
    Since the regex is file-specific, the regex format plugin is most useful if it can be configured per-file using table functions. See [DRILL-6167](https://issues.apache.org/jira/browse/DRILL-6167), [DRILL-6168](https://issues.apache.org/jira/browse/DRILL-6168) and [DRILL-6169](https://issues.apache.org/jira/browse/DRILL-6169) for problems that will be encountered. See the example and tests above for how to work around the bugs.
    
    Although I suggested looking at the `ResultSetLoader`, it is a bit premature to do so. That mechanism relies on additional mechanisms that have not yet been committed to master. So, we need to work with the mechanisms we have now. See the example reader above for how to cache the per-column mutator needed to write to vectors without the switch statements in the code in this PR.
    
    To generalize, the example has an object per column that holds per-column info. The example uses only `VarChar` columns. To add additional types, create a base column state class with subclasses for each type. Then, simply call a `save(String value)` method to write a column. That method can handle nulls (for projected non-existent columns) and type conversions (where needed).
    
    Finally, feel free to borrow liberally from the example. (The example was created for our Drill book, so is fair game to reuse.)


---