You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/05/05 05:41:00 UTC
[jira] [Commented] (IMPALA-10482) Select-star query on unrelative
collection column of transactional table hits IllegalStateException
[ https://issues.apache.org/jira/browse/IMPALA-10482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339425#comment-17339425 ]
ASF subversion and git services commented on IMPALA-10482:
----------------------------------------------------------
Commit f0f083e45e2c77b1499fa6fa08ff8d9dc4a2785f in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f0f083e ]
IMPALA-10482, IMPALA-10493: Fix bugs in full ACID collection query rewrites
IMPALA-10482: SELECT * query on unrelative collection column of
transactional ORC table will hit IllegalStateException.
The AcidRewriter will rewrite queries like
"select item from my_complex_orc.int_array" to
"select item from my_complex_orc t, t.int_array"
This cause troubles in star expansion. Because the original query
"select * from my_complex_orc.int_array" is analyzed as
"select item from my_complex_orc.int_array"
But the rewritten query "select * from my_complex_orc t, t.int_array" is
analyzed as "select id, item from my_complex_orc t, t.int_array".
Hidden table refs can also cause issues during regular column
resolution. E.g. when the table has top-level 'pos'/'item'/'key'/'value'
columns.
The workaround is to keep track of the automatically added table refs
during query rewrite. So when we analyze the rewritten query we can
ignore these auxiliary table refs.
IMPALA-10493: Using JOIN ON syntax to join two full ACID collections
produces wrong results.
When AcidRewriter.splitCollectionRef() creates a new collection ref
it doesn't copy every information needed to correctly execute the
query. E.g. it dropped the ON clause, turning INNER joins to CROSS
joins.
Testing:
* added e2e tests
Change-Id: I8fc758d3c1e75c7066936d590aec8bff8d2b00b0
Reviewed-on: http://gerrit.cloudera.org:8080/17038
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Select-star query on unrelative collection column of transactional table hits IllegalStateException
> ---------------------------------------------------------------------------------------------------
>
> Key: IMPALA-10482
> URL: https://issues.apache.org/jira/browse/IMPALA-10482
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Quanlong Huang
> Assignee: Zoltán Borók-Nagy
> Priority: Critical
>
> {{SELECT *}} query on unrelative collection column of transactional ORC table will hit IllegalStateException.
> Reproduce the bug by:
> {code:sql}
> create table my_complex_orc (id int, int_array array<int>) stored as orc tblproperties('transactional'='true');
> select * from my_complex_orc.int_array;
> {code}
> FE stacktrace:
> {code:java}
> I0206 16:04:42.212499 15294 Frontend.java:1587] 7e42f06526f5791a:e18eb18e00000000] Analyzing query: select * from my_complex_orc.int_array db: default
> I0206 16:04:42.213887 15294 jni-util.cc:288] 7e42f06526f5791a:e18eb18e00000000] java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at org.apache.impala.analysis.StatementBase.castResultExprs(StatementBase.java:114)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:561)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:445)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1627)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1594)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1564)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:159)
> {code}
> cc [~boroknagyz]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org