[jira] [Created] (SPARK-58631) array_repeat and array_insert raise inconsistent array size limit errors in codegen and interpreted execution

["sepuri sai krishna (Jira)" <ji...@apache.org>]

sepuri sai krishna created SPARK-58631:
------------------------------------------

             Summary: array_repeat and array_insert raise inconsistent array size limit errors in codegen and interpreted execution
                 Key: SPARK-58631
                 URL: https://issues.apache.org/jira/browse/SPARK-58631
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 4.2.0
            Reporter: sepuri sai krishna


{{array_repeat}} and {{array_insert}} raise a different error for an array that exceeds the maximum array length depending on whether the expression runs through the interpreted path or through generated code.

h3. array_repeat

{code}
eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER   (SQLSTATE 54000)
codegen -> _LEGACY_ERROR_TEMP_2176
{code}

{{ArrayRepeat.eval}} checks {{count}} against {{ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH}} and raises {{COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER}}. {{ArrayRepeat.genCodeForNumberOfElements}} performs no such check, so the generated code proceeds to the array allocation, which fails with the internal {{_LEGACY_ERROR_TEMP_2176}}. Codegen is the default execution path, so the internal error class is what users normally see. The interpreted path was migrated to the named condition by SPARK-45710 (4.0.0); the codegen path was not.

h3. array_insert

{code}
eval    -> COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION
codegen -> COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER
{code}

The codegen branch taken when {{pos}} is not foldable calls {{createArrayWithElementsExceedLimitError}}, which hardcodes {{"parameter" -> toSQLId("count")}}. The resulting message reads "the value of parameter(s) `count` in the function `array_insert` is invalid", but {{array_insert(array, pos, value)}} has no {{count}} parameter. Every other site in the expression -- all three interpreted ones and the two sibling codegen ones -- uses {{arrayFunctionWithElementsExceedLimitError}} ({{COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION}}). Introduced by SPARK-56567 (4.2.0).

h3. Why this was not caught

The existing test "Elements exceed limit for array_repeat()" in {{QueryExecutionErrorsSuite}} runs {{select array_repeat(1, 2147483647)}}, which is constant-folded and therefore only exercises the interpreted path. The {{array_insert}} branch is reachable only with a non-foldable {{pos}}, because a foldable positive {{pos}} takes a different codegen branch.

h3. Scope

Error reporting only -- neither function produces wrong results, and the conditions under which each raises an error are unchanged.

{{Sequence}} also reports {{COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER}} naming a {{count}} parameter it does not have, but it does so from {{sequenceLength}}, which both execution paths share. There is no interpreted-vs-codegen inconsistency there, so it is left alone.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org