You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "devin-petersohn (via GitHub)" <gi...@apache.org> on 2026/05/05 18:23:20 UTC
[PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
devin-petersohn opened a new pull request, #55690:
URL: https://github.com/apache/spark/pull/55690
### What changes were proposed in this pull request?
Implement `Rolling.median()` and `Expanding.median()` in the pandas API on Spark
### Why are the changes needed?
Missing API coverage
### Does this PR introduce _any_ user-facing change?
Yes
### How was this patch tested?
Unit tests
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-4-6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #55690:
URL: https://github.com/apache/spark/pull/55690#discussion_r3192097922
##########
python/pyspark/pandas/window.py:
##########
@@ -674,6 +677,64 @@ def quantile(self, quantile: float, accuracy: int = 10000) -> FrameLike:
"""
return super().quantile(quantile, accuracy)
+ def median(self) -> FrameLike:
+ """
+ Calculate the rolling median.
+
+ .. versionadded:: 4.2.0
Review Comment:
Let's do 4.3.0. We cut branch-4.2 and we freeze the branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #55690: [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark
URL: https://github.com/apache/spark/pull/55690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #55690:
URL: https://github.com/apache/spark/pull/55690#discussion_r3192094596
##########
python/pyspark/pandas/window.py:
##########
@@ -110,6 +110,9 @@ def quantile(scol: Column) -> Column:
return self._apply_as_series_or_frame(quantile)
+ def median(self) -> FrameLike:
+ return self.quantile(0.5)
Review Comment:
Can we use `approx_percentile` instead?
##########
python/pyspark/pandas/window.py:
##########
@@ -110,6 +110,9 @@ def quantile(scol: Column) -> Column:
return self._apply_as_series_or_frame(quantile)
+ def median(self) -> FrameLike:
+ return self.quantile(0.5)
Review Comment:
Can we use `approx_percentile` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #55690:
URL: https://github.com/apache/spark/pull/55690#issuecomment-4392577022
Merged to master and branch-4.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #55690:
URL: https://github.com/apache/spark/pull/55690#issuecomment-4383827628
Merged to master and branch-4.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org