[jira] [Commented] (IMPALA-15141) Enable codegen for VARIANTs

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18122248#comment-18122248 ] 

ASF subversion and git services commented on IMPALA-15141:
----------------------------------------------------------

Commit 55e46a6739de79fdad885951c140c15e9ac7eb85 in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=55e46a673 ]

IMPALA-15141 (part 1): Keep codegen enabled for queries on VARIANT

The planner set the query-wide disable_codegen_hint for every query
that scans a VARIANT column, so these queries lost codegen in all of
their fragments, even in operators that never touch VARIANT values.
This patch removes the hint and makes the backend safe without it.
Expressions that involve VARIANT are still interpreted.

The hint also protected codegen paths that cannot deal with VARIANT:
* Exec nodes call ScalarExpr::GetCodegendComputeFn() directly (e.g.
  for conjuncts, hash/sort keys and partition exprs), bypassing
  ShouldCodegen(). GetCodegendComputeFn() now returns an expected
  error for expressions that involve VARIANT, so these callers fall
  back to interpretation, but only for the given operator.
* LlvmCodeGen::GetSlotType() now knows the layout of VARIANT slots
  (two StringValues), so TupleDescriptor::GetLlvmStruct() works for
  tuples that have a VARIANT slot.
* Only the Parquet scanner can read VARIANT, but Iceberg tables can
  also have Avro and ORC data files with VARIANT columns. The planner
  now rejects scans that read VARIANT from non-Parquet data files,
  similarly to UUID (IMPALA-15200).

Tuple::CodegenMaterializeExprs() still falls back if the destination
tuple has a VARIANT slot (TOP-N, UNION ALL).

Testing:
* added e2e tests to check that the planner does not disable codegen
* added Iceberg V3 test tables with VARIANT in Avro, ORC and mixed
  data files, generated with the Iceberg Java API

Change-Id: I6e3c59cc41bab7421e0dd450e08944ffc3889241
Assisted-by: Claude Fable 5.1 (Claude Code)
Reviewed-on: http://gerrit.cloudera.org:8080/24882
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Enable codegen for VARIANTs
> ---------------------------
>
>                 Key: IMPALA-15141
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15141
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Zoltán Borók-Nagy
>            Priority: Major
>              Labels: impala-iceberg, impala-iceberg-active-backlog
>
> Currently codegen is disabled when query has VARIANT data. We should add codegen support for VARIANTs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org