You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "AMC-hawk (via GitHub)" <gi...@apache.org> on 2026/09/22 08:22:20 UTC
[PR] [SPARK-59628][SQL] Report a missing MATCH_CONDITION in ASOF JOIN instead of parsing ASOF as an alias [spark]
AMC-hawk opened a new pull request, #58966:
URL: https://github.com/apache/spark/pull/58966
### What changes were proposed in this pull request?
`MATCH_CONDITION` is required for an ASOF join, but `ASOF` is a non-reserved keyword, so it is also a legal table alias. When `MATCH_CONDITION` is missing, the ASOF branch of `joinRelation` cannot match, and the parser falls back to reading `ASOF` as the alias of the left relation:
```sql
SELECT * FROM t ASOF JOIN u ON t.a = u.a;
```
```
'Project [*]
+- 'Join Inner, ('t.a = 'u.a)
:- 'SubqueryAlias asof
: +- 'UnresolvedRelation [t]
+- 'UnresolvedRelation [u]
```
This PR makes that case fail with a clear error, in two parts:
1. `tableAlias` no longer takes `ASOF` as an implicit alias when the next token is `JOIN`. This uses a semantic predicate, `isAsofJoinAhead()`, in the same way the grammar already uses `isOperatorPipeStart()`. An explicit `AS` is unaffected, so `t AS asof JOIN u` still aliases the relation, and `asof` remains usable as an identifier everywhere else.
2. `MATCH_CONDITION` is made optional in `asofJoinCriteria`, so that omitting it reaches `AstBuilder.withAsOfJoin`, which raises the new `ASOF_JOIN_MATCH_CONDITION_MISSING` error (SQLSTATE `42601`). It is still required: omitting it always fails. The check runs after the existing feature flag check, so with ASOF disabled users still see `UNSUPPORTED_FEATURE.ASOF_JOIN` first.
Part 1 alone would turn the silent inner join into a generic `PARSE_SYNTAX_ERROR`; part 2 makes the error name the missing clause, as the JIRA asks.
Making `ASOF` a reserved keyword would also fix the ambiguity, but would break every existing query that uses `asof` as an identifier. The predicate only refuses the one ambiguous shape, an implicit alias immediately before `JOIN`.
### Why are the changes needed?
An ASOF join returns the single nearest right row for each left row. A plain inner join returns every matching row. Parsing a malformed ASOF join as an inner join therefore returns silently wrong results, often many more rows on time series data, with no error or warning.
The behavior was also inconsistent. Only the default join type with an unaliased left relation was misread; with an alias or with `LEFT`, the same mistake failed with a generic `PARSE_SYNTAX_ERROR ... at or near 'ON'` that did not mention `MATCH_CONDITION`.
This is one of the two parser stage defects tracked by SPARK-59626.
### Does this PR introduce _any_ user-facing change?
Yes. An ASOF join without `MATCH_CONDITION` now fails with a clear error in every form, instead of running as an inner join or failing with a generic syntax error. The ASOF JOIN syntax is disabled by default (`spark.sql.join.asofJoin.enabled`) and is unreleased.
With ASOF enabled:
- `SELECT * FROM t ASOF JOIN u ON t.a = u.a`: before, ran as `t AS asof` INNER JOIN `u`. After, `ASOF_JOIN_MATCH_CONDITION_MISSING`.
- `SELECT * FROM t ASOF JOIN u`: before, ran as `t AS asof` INNER JOIN `u`. After, `ASOF_JOIN_MATCH_CONDITION_MISSING`.
- `SELECT * FROM t x ASOF JOIN u ON t.a = u.a`: before, `PARSE_SYNTAX_ERROR` at `'ON'`. After, `ASOF_JOIN_MATCH_CONDITION_MISSING`.
- `SELECT * FROM t LEFT ASOF JOIN u ON t.a = u.a`: before, `PARSE_SYNTAX_ERROR` at `'ON'`. After, `ASOF_JOIN_MATCH_CONDITION_MISSING`.
```
[ASOF_JOIN_MATCH_CONDITION_MISSING] ASOF JOIN requires a MATCH_CONDITION clause, for example MATCH_CONDITION (left.timestamp >= right.timestamp). SQLSTATE: 42601
```
Unchanged: valid ASOF joins; `t AS asof JOIN u ...` and ``t `asof` JOIN u ...``, which still alias the relation as `asof`; `FROM t asof`; and `asof` as a column name. A query that used `asof` as an implicit alias directly before `JOIN` must now write `AS asof` or quote it.
### How was this patch tested?
New tests:
- `PlanParserSuite`: `asof join - missing match condition rejected` covers five shapes (unaliased with `ON`, aliased with `ON`, `USING`, `LEFT` with no condition, no condition at all) and checks the error condition, SQLSTATE and query context. `asof join - an alias named asof is still allowed when written with AS` checks that `t as asof join u ...` and `t asof` still parse to an aliased relation.
- `join-asof-grammar.sql`: three new golden queries for the unaliased case, the case with no join condition, and `trades AS asof JOIN quotes ...` returning rows. The existing aliased case now reports `ASOF_JOIN_MATCH_CONDITION_MISSING` instead of `PARSE_SYNTAX_ERROR`; this is the only change to existing expected output. Golden files were regenerated with `SPARK_GENERATE_GOLDEN_FILES=1`.
Existing tests, since `tableAlias` is used by almost every query:
```
build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.parser.*' # 784 passed
build/sbt 'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite' # 845 passed
build/sbt 'core/testOnly org.apache.spark.SparkThrowableSuite' # 38 passed
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org