You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/11/10 00:37:00 UTC
[jira] [Commented] (IMPALA-11692) Struct slot memory sharing involving select * not working properly
[ https://issues.apache.org/jira/browse/IMPALA-11692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631358#comment-17631358 ]
ASF subversion and git services commented on IMPALA-11692:
----------------------------------------------------------
Commit efa426453a8af3728bc272b9158f5564ce37e0ea in impala's branch refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=efa426453 ]
IMPALA-11692: Struct slot memory sharing involving select * not working properly
With EXPAND_COMPLEX_TYPES=1, if there are structs coming from the star
expansion and members of the structs are also given explicitly, slot
memory sharing does not work in some cases:
explain select * from functional_orc_def.complextypes_nested_structs;
row-size=64B
explain select *, outer_struct.inner_struct1 from
functional_orc_def.complextypes_nested_structs;
row-size=80B
The row size should be the same in both cases as
outer_struct.inner_struct1 is part of outer_struct which is included in
the star.
This change modifies how star select list items are analysed. First,
before 'SelectAnalyzer.analyzeSelectClause()' is called, all star items
are expanded to paths which are stored in the 'starExpandedPaths_' map.
The function 'SelectAnalyzer.registerStructSlotRefPathsWithAnalyzer()',
which makes struct slot memory sharing possible, takes these star
expanded paths into account. Then in
'SelectAnalyzer.analyzeSelectClause()' the paths expanded from the star
items are retrieved and and normal analysis takes place.
Testing:
- Added the test function
PlannerTest.testStructFieldSlotSharedWithStructFromStarExpansion()
that verifies that struct slot memory sharing takes place.
Change-Id: I346c2808c1aa5e77e3cdf3593f7f48ac96516c00
Reviewed-on: http://gerrit.cloudera.org:8080/19190
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Struct slot memory sharing involving select * not working properly
> -------------------------------------------------------------------
>
> Key: IMPALA-11692
> URL: https://issues.apache.org/jira/browse/IMPALA-11692
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Major
>
> With EXPAND_COMPLEX_TYPES=1, if there are structs coming from the star expansion and members of the structs are also given explicitly, slot memory sharing does not work in some cases:
> {code:java}
> explain select * from functional_orc_def.complextypes_nested_structs;
> row-size=64B{code}
> {code:java}
> explain select *, outer_struct.inner_struct1 from functional_orc_def.complextypes_nested_structs;
> row-size=80B{code}
> The row size should be the same in both cases as outer_struct.inner_struct1 is part of outer_struct which is included in the star.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org