[jira] [Assigned] (IMPALA-15434) [DOCS] Document the VARIANT data type and the variant_get() functions

["Zoltán Borók-Nagy (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zoltán Borók-Nagy reassigned IMPALA-15434:
------------------------------------------

    Assignee: Zoltán Borók-Nagy

> [DOCS] Document the VARIANT data type and the variant_get() functions
> ---------------------------------------------------------------------
>
>                 Key: IMPALA-15434
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15434
>             Project: IMPALA
>          Issue Type: Documentation
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Zoltán Borók-Nagy
>            Priority: Major
>              Labels: impala-iceberg
>
> Impala 5.0 reads VARIANT columns of Iceberg tables (IMPALA-15052), hardened the decoder (IMPALA-15221) and now has VARIANT functions (IMPALA-15057). There is no VARIANT content in docs/ at all.
> Create docs/topics/impala_variant.xml ("VARIANT Data Type (Impala 5.0 or higher only)"), add a <topicref> after impala_varchar.xml under impala_datatypes.xml in docs/impala.ditamap and a keydef in docs/impala_keydefs.ditamap. Add a variant row to the "Supported Data Types for Iceberg Columns" table in impala_iceberg.xml — it currently
> ends at map, so readers assume VARIANT is unsupported.
> The page should cover:
> - Read-only support for top-level, unshredded VARIANT columns in Iceberg tables with Parquet data files, as written by engines such as Spark or Trino. Shredded values fail at scan time.
> - Values come back as JSON text: SQL NULL versus a variant null ('null'), quoted strings, base64 for BINARY, quoted DATE and TIMESTAMP, and "<unsupported-type>" for TIME, TIMESTAMPTZ and UUID. Clients see STRING; over JDBC it is VARCHAR with TYPE_NAME 'VARIANT'.
> - Functions (IMPALA-15057): variant_get(VARIANT, path) -> VARIANT (chainable), the typed variant_get(VARIANT, path, 'type'), the try_variant_get() forms, and variant_to_json(VARIANT) -> STRING. Document the path syntax, the coercion rules of the 3-arg form (widening, checked narrowing, string parsing, and DATE only from a real DATE
>   value), and that any query with VARIANT expressions runs interpreted, without codegen.
> - Restrictions: VARIANT can't be used in CREATE TABLE, ALTER TABLE ADD COLUMN or CAST ("VARIANT type is not supported as a column type."); no INSERT into a table with a VARIANT column, so CTAS and INSERT ... SELECT fail; and VARIANT is rejected in ORDER BY, GROUP BY, SELECT DISTINCT, UNION/INTERSECT/EXCEPT, CASE/DECODE, analytic
>   PARTITION BY and ORDER BY, aggregate DISTINCT and IS [NOT] NULL. The current message is "<clause> '<expr>' with type 'VARIANT' is not supported." UNION ALL, joins, inline views and WITH work.
> - A VARIANT nested inside an ARRAY, MAP or STRUCT can't be queried, though DESCRIBE works. VARIANT is not a complex type, so EXPAND_COMPLEX_TYPES doesn't control it and SELECT * always returns it.
> - Limits from IMPALA-15221: a value nested more than 100 levels deep fails; a DATE outside 0001-01-01..9999-12-31 or a TIMESTAMP outside 1400-01-01..9999-12-31 inside a VARIANT fails the query instead of returning NULL (this can happen with valid files from other engines); corrupt encodings fail with "Malformed variant ...".
> - COMPUTE STATS skips VARIANT columns. SHOW CREATE TABLE prints v VARIANT, but that output can't be re-run because VARIANT is rejected in DDL. HMS stores the column as struct<metadata:binary,value:binary>, which matters for interop.
> VARIANT is a non-reserved keyword, so no incompatible-changes entry and no reserved-words change are needed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org