You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Anton Gozhiy (JIRA)" <ji...@apache.org> on 2019/06/25 13:36:00 UTC
[jira] [Comment Edited] (DRILL-6952) Merge row set based
"compliant" text reader
[ https://issues.apache.org/jira/browse/DRILL-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872306#comment-16872306 ]
Anton Gozhiy edited comment on DRILL-6952 at 6/25/19 1:35 PM:
--------------------------------------------------------------
Verified with Drill version 1.17.0-SNAPSHOT (commit f3f7dbd40f5e899f2aacba35db8f50ffedfa9d3d)
Cases checked:
Tested with different storage plugin parameters (extractHeader, delimiters etc.)
The same with table function.
Complex json files with nesting maps and arrays.
Data with implicit columns (with v3 reader, all such columns are moved to the end of rows)
Aggregate functions with specific columns and wildcard.
Large text fields (they was limited to 65536 symbols, now fixed)
No significant changes in performance were discovered. (Compared test runs with different readers)
Some bugs were fixed by V3 reader:
DRILL-5487, DRILL-5554, DRILL-5555 (partially fixed), DRILL-4814, DRILL-7034, DRILL-7082, DRILL-7083
Bugs that were introduced by V3 reader and then fixed:
DRILL-7181, DRILL-7257, DRILL-7258
was (Author: angozhiy):
Verified with Drill version 1.17.0-SNAPSHOT (commit f3f7dbd40f5e899f2aacba35db8f50ffedfa9d3d)
Cases checked:
Tested with different storage plugin parameters (extractHeader, delimiters etc.)
The same with table function.
Complex json files with nesting maps and arrays.
Data with implicit columns (with v3 reader, all such columns are moved to the end of rows)
Aggregate functions with specific columns and wildcard.
Large text fields (they was limited to 65536 symbols, now fixed)
No significant changes in performance were discovered. (Compared test runs with different readers)
Some bugs were fixed by V3 reader:
DRILL-5487, DRILL-5554, DRILL-5555, DRILL-4814, DRILL-7034, DRILL-7082, DRILL-7083
Bugs that were introduced by V3 reader and then fixed:
DRILL-7181, DRILL-7257, DRILL-7258
> Merge row set based "compliant" text reader
> -------------------------------------------
>
> Key: DRILL-6952
> URL: https://issues.apache.org/jira/browse/DRILL-6952
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.15.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
> Labels: ready-to-commit
> Fix For: 1.16.0
>
>
> The result set loader project created a revised version of the compliant text reader that uses the result set loader framework (which includes the schema-based projection framework.)
> This task merges that work into master:
> * Review the history of the complaint text reader for changes made in the last year since the code was written.
> * Apply those changes to the row set-based code, as necessary.
> * Issue a PR for the new version of the compliant text reader
> * Work through any test issues that crop up in the pre-commit tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)