[jira] [Commented] (IMPALA-11685) Slot memory sharing between struct and field not working if the field is also a struct

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17625199#comment-17625199 ] 

ASF subversion and git services commented on IMPALA-11685:
----------------------------------------------------------

Commit ca9b08372556ac2010348b7c978c350e538b7b2c in impala's branch refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ca9b08372 ]

IMPALA-11685: Slot memory sharing between struct and field not working
if the field is also a struct

IMPALA-10838 introduced that if a struct and one of its fields are both
present in the select list, no extra slot is generated in the row for
the struct field but the memory of the struct is reused, i.e. the row
size is the same as when only the struct is queried. It works when the
struct field is a primitive type:

explain select id, outer_struct from
functional_orc_def.complextypes_nested_structs;
row-size=64B

explain select id, outer_struct, outer_struct.str from
functional_orc_def.complextypes_nested_structs;
row-size=64B

However, it does not if the child is itself a struct:

explain select id, outer_struct, outer_struct.inner_struct3 from
functional_orc_def.complextypes_nested_structs;
row-size=80B

This is because struct slot descriptors are registered before others so
that it is easier to reuse the slot memory of the struct fields, but
struct slot descriptors among themselves are sorted in the wrong order
(see
https://github.com/apache/impala/blob/c12ac6c27b2df1eae693b44c157d65499f491d21/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java#L340).

Also, because struct children are also inserted into the Analyzer's
slotPathMap_ when the struct is registered, there is no need for the
mechanism that tries to find ancestors of fields to be able to share
their slot memory - they can be retrieved from the slotPathMap_. This
change deletes the code that dealt with that.

Testing:
 - PlannerTest#testStructFieldSlotSharedWithStruct has been updated to
   include queries where the struct field is also a struct; the elements
   in the select list are permutated to make sure the order in which
   they are listed does not matter.

Change-Id: I6d4dee3941fb2d285fbd3836ea5712c859db8848
Reviewed-on: http://gerrit.cloudera.org:8080/19167
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Slot memory sharing between struct and field not working if the field is also a struct
> --------------------------------------------------------------------------------------
>
>                 Key: IMPALA-11685
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11685
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>            Reporter: Daniel Becker
>            Assignee: Daniel Becker
>            Priority: Major
>
> IMPALA-10838 introduced that if a struct and one of its fields are both present in the select list, no extra slot is generated in the row for the struct field but the memory of the struct is reused, i.e. the row size is the same as when only the struct is queried. It works when the struct field is a primitive type:
> {code:java}
> explain select id, outer_struct from functional_orc_def.complextypes_nested_structs;
> row-size=64B{code}
> {code:java}
> explain select id, outer_struct, outer_struct.str from functional_orc_def.complextypes_nested_structs;
> row-size=64B{code}
> However, it does not if the child is itself a struct:
> {code:java}
> explain select id, outer_struct, outer_struct.inner_struct3 from functional_orc_def.complextypes_nested_structs;
> row-size=80B{code}
> This is because struct slot descriptors are registered before others so that it is easier to reuse the slot memory of the struct fields, but struct slot descriptors among themselves are sorted in the wrong order (see [https://github.com/apache/impala/blob/c12ac6c27b2df1eae693b44c157d65499f491d21/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java#L340).]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org