[jira] [Commented] (DRILL-7240) Run-time rowgroup pruning match() fails on casting a Long to an Integer

["Boaz Ben-Zvi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832964#comment-16832964 ] 

Boaz Ben-Zvi commented on DRILL-7240:
-------------------------------------

Example recreating this bug - take this json file
{noformat}
{"key": "aa", "sales": 11}
{"key": "bb", "sales": 22}
{noformat}
And create two parquet tables/files by selecting from the json, first casting the "sales" to an INT, and the second to a BIGINT:
{noformat}
create table test_int as select key, cast(sales as int) sales from dfs.`/tmp/myfile.json`;
create table test_bigint as select key, cast(sales as bigint) sales from dfs.`/tmp/myfile.json`;
{noformat}
Then move the two files into a sub-directory, renaming the second:
{noformat}
$ > mv /tmp/test_int/0_0_0.parquet /tmp/test/sub
$ > mv /tmp/test_bigint/0_0_0.parquet /tmp/test/sub/0_0_1.parquet 
{noformat}
Last refresh on only the first "key" columns, then run a query with a predicate on the 'sales" column:
{noformat}
refresh table METADATA columns(key) dfs.`/tmp/test`;
select sales from dfs.`/tmp/test/` where sales > 10;
{noformat}


> Run-time rowgroup pruning match() fails on casting a Long to an Integer
> -----------------------------------------------------------------------
>
>                 Key: DRILL-7240
>                 URL: https://issues.apache.org/jira/browse/DRILL-7240
>             Project: Apache Drill
>          Issue Type: Sub-task
>          Components: Storage - Parquet
>    Affects Versions: 1.17.0
>            Reporter: Boaz Ben-Zvi
>            Assignee: Boaz Ben-Zvi
>            Priority: Major
>             Fix For: 1.17.0
>
>
> After a Parquet table is refreshed with selected "interesting" columns, a query whose WHERE clause contains a condition on a "non interesting" INT64 column fails during run-time pruning (calling match()) with:
> {noformat}
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
> {noformat}
>  Near-term fix suggestion: Catch the match() exception error, and instead do not prune (i.e. run-time pruning would be disabled in such cases).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)