You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ur...@apache.org on 2026/07/22 15:50:09 UTC
(spark) branch branch-4.x updated: [MINOR][PYTHON][DOC] Add example to Column.outer docstring
This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 665032c40e0f [MINOR][PYTHON][DOC] Add example to Column.outer docstring
665032c40e0f is described below
commit 665032c40e0ff9ccb339539d2ed89d2139760624
Author: eunsang <th...@gmail.com>
AuthorDate: Wed Jul 22 15:49:39 2026 +0000
[MINOR][PYTHON][DOC] Add example to Column.outer docstring
### What changes were proposed in this pull request?
Adds an `Examples` section to `Column.outer()`'s docstring in `python/pyspark/sql/column.py`. Every other `Column` method in this file has a worked example; `outer()` was the one exception.
The example builds a small `employees` DataFrame and uses `.outer()` inside a correlated scalar subquery to filter employees earning above their department's average salary.
### Why are the changes needed?
`outer()` is easy to misunderstand from the description alone, since its effect (marking a column so it resolves against an outer query) only makes sense in the context of a correlated subquery. A concrete example makes that immediately clear, consistent with the rest of the file.
### Does this PR introduce _any_ user-facing change?
No. Documentation-only change (a docstring example).
### How was this patch tested?
Built Spark locally (`build/sbt -Phive package`) and ran the `Column.outer` doctest directly against the built jars to confirm the example's output matches exactly.
### Was this patch authored or co-authored using generative AI tooling?
Yes, co-authored. The code example and documentation text were authored by me, but I used Claude as a peer-review to validate the clarity of the example and check for edge cases before submission.
Closes #57199 from CynicDog/column-outer-doc-examples.
Authored-by: eunsang <th...@gmail.com>
Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
(cherry picked from commit 54f21fbbf1e6404b02e5f76472e327fa1375a920)
Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
---
python/pyspark/sql/column.py | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/python/pyspark/sql/column.py b/python/pyspark/sql/column.py
index af25fd1b77e1..7c39a0ffbc33 100644
--- a/python/pyspark/sql/column.py
+++ b/python/pyspark/sql/column.py
@@ -1578,6 +1578,28 @@ class Column(TableValuedFunctionArgument):
--------
pyspark.sql.DataFrame.scalar
pyspark.sql.DataFrame.exists
+
+ Examples
+ --------
+ >>> from pyspark.sql import functions as sf
+ >>> employees = spark.createDataFrame(
+ ... [
+ ... (1, "Alice", 45000, 101), (2, "Bob", 54000, 101), (3, "Charlie", 29000, 102),
+ ... (4, "David", 61000, 102), (5, "Eve", 48000, 101),
+ ... ],
+ ... ["id", "name", "salary", "department_id"],
+ ... )
+ >>> employees.alias("e1").where(
+ ... sf.col("salary") > employees.alias("e2").where(
+ ... sf.col("e2.department_id") == sf.col("e1.department_id").outer()
+ ... ).select(sf.avg("salary")).scalar()
+ ... ).select("name", "salary", "department_id").orderBy("name").show()
+ +-----+------+-------------+
+ | name|salary|department_id|
+ +-----+------+-------------+
+ | Bob| 54000| 101|
+ |David| 61000| 102|
+ +-----+------+-------------+
"""
...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org