You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/07/22 04:57:28 UTC
[PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
LuciferYang opened a new pull request, #57423:
URL: https://github.com/apache/spark/pull/57423
### What changes were proposed in this pull request?
An out-of-range GROUP BY ordinal in the SQL pipe AGGREGATE operator threw a raw `java.lang.IndexOutOfBoundsException` instead of a proper `AnalysisException`. For example:
```sql
select 3 as x, 4 as y
|> aggregate sum(y) group by 5;
```
`resolvePipeAggregateExpressionOrdinal` resolved the ordinal with an unguarded `inputs(index - 1)`, so an out-of-range position (or `0`) surfaced the internal JVM exception.
This PR adds the same bounds guard the regular GROUP BY ordinal resolver (`resolveGroupByExpressionOrdinal`) already uses, throwing `GROUP_BY_POS_OUT_OF_RANGE` (sqlState `42805`). The `UnresolvedPipeAggregateOrdinal` node is now created with the ordinal literal's origin (via `CurrentOrigin.withOrigin`), so the error's `queryContext` points at the offending ordinal rather than the whole pipe statement, matching regular GROUP BY.
### Why are the changes needed?
A raw `IndexOutOfBoundsException` escaping the analyzer for valid-syntax SQL is an internal error leak. User-facing errors should carry an error class, sqlState, and query context. The pipe AGGREGATE path was the only pipe operator with its own GROUP BY ordinal handling and the only one missing this guard; regular `GROUP BY <ordinal>` has thrown `GROUP_BY_POS_OUT_OF_RANGE` with precise context for years.
### Does this PR introduce _any_ user-facing change?
Yes, on the error path only. For an out-of-range or non-positive ordinal in pipe AGGREGATE GROUP BY:
Before:
```
java.lang.IndexOutOfBoundsException: Index 4 out of bounds for length 2
```
After:
```
[GROUP_BY_POS_OUT_OF_RANGE] GROUP BY position 5 is not in select list (valid range is [1, 2]). SQLSTATE: 42805
```
with a query context pointing at the ordinal. Valid (in-range) ordinals are unaffected.
### How was this patch tested?
Added positive-range-adjacent negative cases to `pipe-operators.sql` (`group by 5` above range, `group by 0` non-positive) in the aggregation negative-tests section; regenerated the golden files. The regenerated goldens assert the error class, sqlState, `index`/`size` parameters, and the precise `queryContext` (fragment pointing at the ordinal). Existing positive pipe-ordinal cases (including the exact upper bound `group by x, 2, 3`) regenerated with no diff, confirming zero happy-path impact.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #57423:
URL: https://github.com/apache/spark/pull/57423#issuecomment-5053692662
Thank you @dongjoon-hyun and @uros-b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #57423:
URL: https://github.com/apache/spark/pull/57423#issuecomment-5050644630
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/4462cb1dd30ac46ab27bef394c856be1e760757f
- merged into branch-4.x https://github.com/apache/spark/commit/9d4cb165de5683c00ad5d6ff8243151775eedcf8
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #57423:
URL: https://github.com/apache/spark/pull/57423#issuecomment-5043997076
cc @cloud-fan @dtenedor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #57423: [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY
URL: https://github.com/apache/spark/pull/57423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58252][SQL] Throw GROUP_BY_POS_OUT_OF_RANGE for out-of-range ordinal in pipe AGGREGATE GROUP BY [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #57423:
URL: https://github.com/apache/spark/pull/57423#issuecomment-5050651736
Merged for Apache Spark 4.3.0 whose `Feature Freeze` is scheduled on 1st August.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org