You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2026/05/27 06:09:26 UTC
[PR] [PYTHON][DOCS] 4.2 audit follow-ups: document TwsTester, fix kll_merge_agg_* versionadded [spark]
zhengruifeng opened a new pull request, #56135:
URL: https://github.com/apache/spark/pull/56135
### What changes were proposed in this pull request?
Two follow-ups to the 4.2 PySpark API-reference audit (related: #56115, #56116):
1. **Add `TwsTester` to the API reference.** `pyspark.sql.streaming.TwsTester` was added in 4.2 by SPARK-54805 (#53758) but was never registered in the docs. This PR adds a `tws_tester.rst` page (mirroring the existing `stateful_processor.rst` pattern) listing all 11 public methods, and wires it up via `index.rst`.
2. **Fix wrong `versionadded` on `kll_merge_agg_{bigint,float,double}`.** The three aggregation functions are annotated `.. versionadded:: 4.1.0`, but the introducing commit (SPARK-54785, `fc15f726eab` on 2026-01-12) landed on master after v4.1.0 was tagged (2025-12-11). They first appear in `v4.2.0-preview2`, so the annotation is updated to `4.2.0`.
### Why are the changes needed?
(1) Without the autosummary entries, `TwsTester` does not appear in the rendered Python docs even though it is public (re-exported in `pyspark.sql.streaming.__init__`).
(2) The rendered ref pages for the three `kll_merge_agg_*` functions currently say "New in version 4.1.0", which is misleading — no 4.1.x release contains them.
### Does this PR introduce _any_ user-facing change?
Documentation-only change. No behavior change.
### How was this patch tested?
Docs-only change. New `tws_tester.rst` follows the same structure as `stateful_processor.rst`; method list matches the public surface of `TwsTester` in source order.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (model: claude-opus-4-7)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57116][SQL][PYTHON][DOC] Fix versionadded/@since for kll_merge_agg_* (4.1.0 -> 4.1.2) [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #56135:
URL: https://github.com/apache/spark/pull/56135#issuecomment-4559782296
@dongjoon-hyun thanks for pointing it out, filed https://issues.apache.org/jira/browse/SPARK-57116 for this fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [PYTHON][DOCS] Fix versionadded for kll_merge_agg_* (4.1.0 -> 4.2.0) [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #56135: [PYTHON][DOCS] Fix versionadded for kll_merge_agg_* (4.1.0 -> 4.2.0)
URL: https://github.com/apache/spark/pull/56135
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57116][SQL][PYTHON][DOC] Fix versionadded/@since for kll_merge_agg_* (4.1.0 -> 4.1.2) [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #56135:
URL: https://github.com/apache/spark/pull/56135#issuecomment-4561481706
thanks, merged to master/4.x/4.2/4.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57116][SQL][PYTHON][DOC] Fix versionadded/@since for kll_merge_agg_* (4.1.0 -> 4.1.2) [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #56135: [SPARK-57116][SQL][PYTHON][DOC] Fix versionadded/@since for kll_merge_agg_* (4.1.0 -> 4.1.2)
URL: https://github.com/apache/spark/pull/56135
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-54785][SQL][PYTHON][DOCS][FOLLOWUP] Fix versionadded/@since for kll_merge_agg_* (4.1.0 -> 4.1.2) [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #56135:
URL: https://github.com/apache/spark/pull/56135#issuecomment-4552477794
Yeah let's try to avoid backporting them to old branches next time ..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org