[jira] [Assigned] (SPARK-58631) array_repeat and array_insert raise inconsistent array size limit errors in codegen and interpreted execution

["Uroš Bojanić (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-58631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uroš Bojanić reassigned SPARK-58631:
------------------------------------

    Assignee: sepuri sai krishna

> array_repeat and array_insert raise inconsistent array size limit errors in codegen and interpreted execution
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-58631
>                 URL: https://issues.apache.org/jira/browse/SPARK-58631
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 4.2.0
>            Reporter: sepuri sai krishna
>            Assignee: sepuri sai krishna
>            Priority: Minor
>              Labels: pull-request-available
>
> {{array_repeat}} and {{array_insert}} raise a different error for an array that exceeds the maximum array length depending on whether the expression runs through the interpreted path or through generated code.
> h3. array_repeat
> {code}
> eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER   (SQLSTATE 54000)
> codegen -> _LEGACY_ERROR_TEMP_2176
> {code}
> {{ArrayRepeat.eval}} checks {{count}} against {{ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH}} and raises {{COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER}}. {{ArrayRepeat.genCodeForNumberOfElements}} performs no such check, so the generated code proceeds to the array allocation, which fails with the internal {{_LEGACY_ERROR_TEMP_2176}}. Codegen is the default execution path, so the internal error class is what users normally see. The interpreted path was migrated to the named condition by SPARK-45710 (4.0.0); the codegen path was not.
> h3. array_insert
> {code}
> eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION
> codegen -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER
> {code}
> The codegen branch taken when {{pos}} is not foldable calls {{createArrayWithElementsExceedLimitError}}, which hardcodes {{"parameter" -> toSQLId("count")}}. The resulting message reads "the value of parameter(s) `count` in the function `array_insert` is invalid", but {{array_insert(array, pos, value)}} has no {{count}} parameter. Every other site in the expression -- all three interpreted ones and the two sibling codegen ones -- uses {{arrayFunctionWithElementsExceedLimitError}} ({{COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION}}). Introduced by SPARK-56567 (4.2.0).
> h3. Why this was not caught
> The existing test "Elements exceed limit for array_repeat()" in {{QueryExecutionErrorsSuite}} runs {{select array_repeat(1, 2147483647)}}, which is constant-folded and therefore only exercises the interpreted path. The {{array_insert}} branch is reachable only with a non-foldable {{pos}}, because a foldable positive {{pos}} takes a different codegen branch.
> h3. Scope
> Error reporting only -- neither function produces wrong results, and the conditions under which each raises an error are unchanged.
> {{Sequence}} also reports {{COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER}} naming a {{count}} parameter it does not have, but it does so from {{sequenceLength}}, which both execution paths share. There is no interpreted-vs-codegen inconsistency there, so it is left alone.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org