[jira] [Commented] (ASTERIXDB-2933) Support more pushdowns when reading Parquet files

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ASTERIXDB-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17409119#comment-17409119 ] 

ASF subversion and git services commented on ASTERIXDB-2933:
------------------------------------------------------------

Commit ca2a2e034b2b9349f5c3664432cd6ced36c1903a in asterixdb's branch refs/heads/master from Wail Alkowaileet
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=ca2a2e0 ]

[ASTERIXDB-2933][COMP][EXT] Pushdowns Part2: Pushdown Rule

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
Add a rule that computes the expected schema and sets it
to the DataSourceScanOperator. The computed schema is then
passed to the Parquet Reader to 'clip' the Parquet file's
schema. The resulting clipped schema is then used to tell
the reader what column should be read.

Interface changes:
- Change IProjectionInfo<List<T>> to IProjectionInfo<T>

Change-Id: If0c0d05473be72df6f08dfcbab2d25c36c71368e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12964
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Wael Alkowaileet <wa...@gmail.com>
Reviewed-by: Dmitry Lychagin <dm...@couchbase.com>


> Support more pushdowns when reading Parquet files
> -------------------------------------------------
>
>                 Key: ASTERIXDB-2933
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2933
>             Project: Apache AsterixDB
>          Issue Type: Improvement
>          Components: COMP - Compiler, EXT - External data
>    Affects Versions: 0.9.7
>            Reporter: Wail Y. Alkowaileet
>            Assignee: Wail Y. Alkowaileet
>            Priority: Major
>             Fix For: 0.9.7
>
>
> Currently, AsterixDB only supports simple pushdowns for field access. We should allow for more pushdowns including array functions (e.g., array_star).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)