You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ericm-db (via GitHub)" <gi...@apache.org> on 2026/01/06 23:47:06 UTC
[PR] [SPARK-54910][SS] Add sourceIdentifyingName parameter to StreamingRelationV2 [spark]
ericm-db opened a new pull request, #53704:
URL: https://github.com/apache/spark/pull/53704
## What changes were proposed in this pull request?
This PR adds the `sourceIdentifyingName` parameter to `StreamingRelationV2` and introduces the `HasStreamingSourceIdentifyingName` trait to provide a uniform interface for streaming source naming.
Key changes:
- Added `HasStreamingSourceIdentifyingName` trait in `StreamingSourceIdentifyingName.scala`
- Added `sourceIdentifyingName` parameter to `StreamingRelationV2` case class with default value `Unassigned`
- Implemented `withSourceIdentifyingName` method to support name propagation
- Updated all pattern matches for `StreamingRelationV2` in sql/core to account for new parameter
- Updated explain test golden file to include the new parameter in output
## Why are the changes needed?
This change lays the groundwork for streaming source evolution by enabling sources to have stable identifying names. The parameter will be used by analyzer rules to assign and propagate names from DataFrame API calls, allowing sources to maintain their checkpoint state even when sources are added, removed, or reordered in a query.
## Does this PR introduce _any_ user-facing change?
No. The parameter is added with a default value (`Unassigned`) that maintains backward compatibility. No user-facing APIs are modified.
## How was this patch tested?
- Updated ProtoToPlan explain test to verify the parameter appears correctly in explain output
- Verified pattern matches compile correctly with the new parameter
- All existing tests pass with the new parameter
## Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-54910][SS] Add sourceIdentifyingName parameter to StreamingRelationV2 [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #53704:
URL: https://github.com/apache/spark/pull/53704#issuecomment-3716756441
## JIRA Issue Information
=== Task SPARK-54910 ===
Summary: Add streamingSourceIdentifyingName field to StreamingRelationV2
Assignee: None
Status: Open
Affected: ["4.2.0"]
---
*This comment was automatically generated by GitHub Actions*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-54910][SS] Add sourceIdentifyingName parameter to StreamingRelationV2 [spark]
Posted by "ericm-db (via GitHub)" <gi...@apache.org>.
ericm-db commented on PR #53704:
URL: https://github.com/apache/spark/pull/53704#issuecomment-3716757416
cc @HeartSaVioR Can you PTAL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-54910][SS] Add sourceIdentifyingName parameter to StreamingRelationV2 [spark]
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db closed pull request #53704: [SPARK-54910][SS] Add sourceIdentifyingName parameter to StreamingRelationV2
URL: https://github.com/apache/spark/pull/53704
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org