You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/01/23 21:30:22 UTC
[PR] [SPARK-50967] Document the case where keys part of the initial state df are also emitted as part of the output [spark]
anishshri-db opened a new pull request, #49630:
URL: https://github.com/apache/spark/pull/49630
### What changes were proposed in this pull request?
Document the case where keys part of the initial state df are also emitted as part of the output
### Why are the changes needed?
Explain the case where the initial state might generate unexpected output
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Docs only change
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50967] Document the case where keys part of the initial state df are also emitted as part of the output [spark]
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db commented on PR #49630:
URL: https://github.com/apache/spark/pull/49630#issuecomment-2611059427
cc - @HeartSaVioR - PTAL
there are a couple different ways to fix this:
- document the existing behavior as a limitation
- add a new flag and change the behavior if the feature is enabled (which is a moderate'ish change)
Also, with the new `transformWithState` operator not having this issue - I wonder whether we should just document this case as such
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50967][SS] Document the case where keys part of the initial state df are also emitted as part of the output [spark]
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db commented on PR #49630:
URL: https://github.com/apache/spark/pull/49630#issuecomment-2611298601
Discussed offline and we will close this in favor of this PR - https://github.com/apache/spark/pull/49632 Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50967][SS] Document the case where keys part of the initial state df are also emitted as part of the output [spark]
Posted by "anishshri-db (via GitHub)" <gi...@apache.org>.
anishshri-db closed pull request #49630: [SPARK-50967][SS] Document the case where keys part of the initial state df are also emitted as part of the output
URL: https://github.com/apache/spark/pull/49630
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org