[jira] [Commented] (DRILL-8037) Add V2 JSON Format Plugin based on EVF

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438992#comment-17438992 ] 

ASF GitHub Bot commented on DRILL-8037:
---------------------------------------

vdiravka opened a new pull request #2364:
URL: https://github.com/apache/drill/pull/2364


   # [DRILL-8037](https://issues.apache.org/jira/browse/DRILL-8037): Add V2 JSON Format Plugin based on EVF
   
   ## Description
   
   This adds new V2 beta JSON Format Plugin based on the "Extended Vector Framework".
   This is follow up [DRILL-6953](https://issues.apache.org/jira/browse/DRILL-6953) (was closed with the decision to merge it by small pieces).
   So it is based on #1913 and [rev2](https://github.com/paul-rogers/drill/tree/DRILL-6953-rev2) work.
   
   ## Documentation
   The new "V2" JSON scan is controlled by a new option:
   store.json.enable_v2_reader, which is false by default in this PR.
   
   Adds a "projection type" to the column writer so that the
   JSON parser can receive a "hint" as to the expected type.
   The hint is from the form of the projected column: a[0],
   a.b or just a.
   Therefore it supports schema provision. Example:
   ```
   ALTER SESSION SET `store.json.enable_v2_reader` = true;
   apache drill (dfs.tmp)> select * from test;
   +---------------+-------+---+
   |       a       |   e   | f |
   +---------------+-------+---+
   | {"b":1,"c":1} | false | 1 |
   | {"b":1,"c":1} | null  | 2 |
   | {"b":1,"c":1} | true  | 3 |
   +---------------+-------+---+
   apache drill (dfs.tmp)> create or replace schema (`e` BOOLEAN default 'false', `new` VARCHAR not null default 'schema evolution') for table test;
   apache drill (dfs.tmp)> select * from test;
   +-------+------------------+---------------+---+
   |   e   |       new        |       a       | f |
   +-------+------------------+---------------+---+
   | false | schema evolution | {"b":1,"c":1} | 1 |
   | null  | schema evolution | {"b":1,"c":1} | 2 |
   | true  | schema evolution | {"b":1,"c":1} | 3 |
   +-------+------------------+---------------+---+
   ```
   
   
   ## Testing
   A lot of existing test cases are running for both readers. It is needed until enabling V2 by default


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Add V2 JSON Format Plugin based on EVF
> --------------------------------------
>
>                 Key: DRILL-8037
>                 URL: https://issues.apache.org/jira/browse/DRILL-8037
>             Project: Apache Drill
>          Issue Type: Sub-task
>            Reporter: Vitalii Diravka
>            Assignee: Vitalii Diravka
>            Priority: Major
>
> This adds new V2 beta JSON Format Plugin based on the "Extended Vector Framework".
> This is follow up DRILL-6953 (was closed with the decision to merge it by small pieces).
> So it is based on [https://github.com/apache/drill/pull/1913] and [https://github.com/paul-rogers/drill/tree/DRILL-6953-rev2] work.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)