You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "cloud-fan (via GitHub)" <gi...@apache.org> on 2026/03/18 18:51:19 UTC
[PR] [SPARK-XXX][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time [spark]
cloud-fan opened a new pull request, #54888:
URL: https://github.com/apache/spark/pull/54888
### What changes were proposed in this pull request?
This PR adds `withResolvedIdentifierReference` to `AstBuilder`, which eagerly resolves `IDENTIFIER()` calls with string literal arguments at parse time instead of deferring to the analysis phase via `PlanWithUnresolvedIdentifier`.
The method is applied to:
- `createUnresolvedRelation` in `AstBuilder`
- Pipeline dataset identifier resolution in `SparkSqlParser`
### Why are the changes needed?
When `IDENTIFIER()` is called with a string literal (e.g., `IDENTIFIER('db.table')`), the value is already known at parse time. The current implementation always defers resolution to analysis by wrapping it in `PlanWithUnresolvedIdentifier`, which is unnecessary overhead for this case.
`withResolvedIdentifierReference` resolves string literals eagerly at parse time and falls back to the existing `withIdentClause` (deferred resolution) for non-literal expressions.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
### Was this patch authored or co-authored using generative AI tooling?
Yes, Claude Opus 4.6.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56050][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #54888:
URL: https://github.com/apache/spark/pull/54888#issuecomment-4093953683
Merged to master for Apache Spark 4.2.0 because the test coverage or PR description is nit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56050][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #54888:
URL: https://github.com/apache/spark/pull/54888#issuecomment-4097214122
it's a perf improvement so I didn't add a new test. Actually the PR description was generated by claude code, hopefully llm can follow the PR template more strictly after we improving `AGENTS.md` (I didn't explicitly ask llm to follow PR template as I thought it's a common rule).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56050][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #54888: [SPARK-56050][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time
URL: https://github.com/apache/spark/pull/54888
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56050][SQL] Eagerly resolve IDENTIFIER() with string literals at parse time [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #54888:
URL: https://github.com/apache/spark/pull/54888#issuecomment-4091390405
cc @peter-toth for this SQL parser optimization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org