[GitHub] [drill] paul-rogers opened a new pull request #2073: DRILL-7725: Updates to the EVF2 framework

[GitBox <gi...@apache.org>]

paul-rogers opened a new pull request #2073:
URL: https://github.com/apache/drill/pull/2073


   # [DRILL-7725](https://issues.apache.org/jira/browse/DRILL-7725): Updates to the EVF2 framework
   
   ## Description
   
   Includes the refinements needed to move the text (CSV) reader to the upgraded EVF (so-called "EVF2"). To avoid excessive complexity, this PR includes just the EVF2 changes. The text reader integration will appear in a separate PR. At present, no code in master yet uses EVF2.
   
   Supports the internal implicit columns needed for the metadata framework.
   
   Provides better support for standard conversions for text-based readers. A new `FixedReceiver` acts to receive text columns when the schema is known up front, as for the CSV reader. (Future versions will include a `DynamicReceiver` to receive columns by name, as for JSON.)
   
   Based on suggestions from Charles, simplified how we handle files for file format plugins. The scan framework takes care of most of the work. It picks up the "compressible" attribute from the plugin, then offers a simple `open()` method that takes care of opening the file with the Drill file system, using the compressible option as needed. The `SchemaNegotiator` exposes a `FileDescrip` interface that handles opening the file, as well as giving access to the block offset and size, etc.
   
   Handled a number of corner cases when a reader returns no results, but we do have a schema. Differentiated this from the case where the reader is simply empty: no data and no schema.
    
   ## Documentation
   
   N/A
   
   ## Testing
   
   Added new tests and ran all existing tests. In fact, ran all tests with the text reader converted (but the text reader code itself is not included here.)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [drill] asfgit closed pull request #2073: DRILL-7725: Updates to the EVF2 framework

[GitBox <gi...@apache.org>]

asfgit closed pull request #2073:
URL: https://github.com/apache/drill/pull/2073


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org