You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Gabor Kaszab (Jira)" <ji...@apache.org> on 2021/08/04 13:25:00 UTC
[jira] [Comment Edited] (IMPALA-10838) Error when struct returned
from WITH() and used in an ORDER BY
[ https://issues.apache.org/jira/browse/IMPALA-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393146#comment-17393146 ]
Gabor Kaszab edited comment on IMPALA-10838 at 8/4/21, 1:24 PM:
----------------------------------------------------------------
After some poking, it turned out that you don't need an ORDER BY to reproduce this issue:
{code:java}
with sub as (
select id, outer_struct
from functional_orc_def.complextypes_nested_structs)
select sub.id, sub.outer_struct.inner_struct2 from sub;
{code}
{code:java}
ERROR: IllegalStateException: Illegal reference to non-materialized tuple: debugname=InlineViewRef sub alias=sub tid=6
{code}
It seems that if you reference anything struct related from the inline view that is not directly present in the select list of the inline view then Impala tries to use a Slot that is present in a non-materialized tuple descriptor.
Adding 'outer_struct.inner_struct2' to the select list of the inline view will work as a workaround.
was (Author: gaborkaszab):
After some poking, it turned out that you don't need an ORDER BY to reproduce this issue:
{code:java}
with sub as (
select id, outer_struct
from functional_orc_def.complextypes_nested_structs)
select sub.id, sub.outer_struct.inner_struct2 from sub;
{code}
{code:java}
ERROR: IllegalStateException: Illegal reference to non-materialized tuple: debugname=InlineViewRef sub alias=sub tid=6
{code}
It seems that if you reference anything struct related from the inline view that is not directly present in the select list of the inline view then Impala tries to use a Slot that is present in a non-materialized tuple descriptor.
> Error when struct returned from WITH() and used in an ORDER BY
> --------------------------------------------------------------
>
> Key: IMPALA-10838
> URL: https://issues.apache.org/jira/browse/IMPALA-10838
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Gabor Kaszab
> Priority: Major
> Labels: complextype, nested_types
>
> {code:java}
> with sub as (
> select id, small_struct
> from functional_orc_def.complextypes_structs
> where length(small_struct.s) > 5)
> select sub.id, sub.small_struct from sub order by sub.small_struct.i desc;
> {code}
> The above query results an error when trying to run SlotRef.toThrift()
> {code:java}
> ERROR: IllegalStateException: Illegal reference to non-materialized tuple: debugname=InlineViewRef sub alias=sub tid=2
> {code}
> If I rewrite the query a bit to return the member of the struct from the inline view (WITH()) and use this in the ORDER by then the query succeeds as expected:
> {code:java}
> with sub as (
> select id, small_struct, small_struct.i as si
> from functional_orc_def.complextypes_structs where small_struct.i > 19200)
> select sub.id, sub.small_struct from sub order by sub.si desc;
> {code}
> In SortNode.toThrift() I checked what the sort exprs and the resolved tuple exprs are and I see a difference that could be the cause.
> In the problematic case:
> {code:java}
> - sort exprs in SortNode:
> SlotRef{label=small_struct.i, type=INT, id=15}
> - resolved exprs in SortNode:
> SlotRef{label=id, path=id, type=INT, id=0}
> SlotRef{label=small_struct, path=small_struct, type=STRUCT<i:INT,s:STRING>, id=1}
> *SlotRef{label=sub.small_struct.i, path=sub.small_struct.i, type=INT, id=10}*
> {code}
> In the successful case:
> {code:java}
> - sort exprs in SortNode:
> SlotRef{label=si, type=INT, id=14}
> - resolved exprs in SortNode:
> SlotRef{label=id, path=id, type=INT, id=0}
> SlotRef{label=small_struct, path=small_struct, type=STRUCT<i:INT,s:STRING>, id=1}
> *SlotRef{label=small_struct.i, path=small_struct.i, type=INT, id=4}*
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org