You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "jiayuasu (via GitHub)" <gi...@apache.org> on 2026/06/08 18:22:11 UTC
[PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
jiayuasu opened a new pull request, #925:
URL: https://github.com/apache/sedona-db/pull/925
Small follow-up to #908. Completes the join surface from #791.
## API
```python
df.cross_join(other)
```
- Returns the Cartesian product of `self` and `other`.
- Result row count is `len(self) * len(other)`.
- Both sides' columns are kept verbatim — matches PySpark's `crossJoin` shape. Disambiguate via `df.alias(...)` if column names collide.
- No `on=` / `how=` — those would be `join(...)`.
## Implementation
DataFusion's `DataFrame` doesn't expose a direct `cross_join` method (52.5); the builder lives on `LogicalPlanBuilder`. The Rust binding uses `DataFrame::into_parts` to split off the `SessionState`, runs `LogicalPlanBuilder::cross_join`, and reassembles with `DataFrame::new`.
| File | Change |
|---|---|
| `python/sedonadb/src/dataframe.rs` | New `InternalDataFrame::cross_join(right)`. Plan-builder path; no per-row work. |
| `python/sedonadb/python/sedonadb/dataframe.py` | New `DataFrame.cross_join(other)`. Validates `other` is a DataFrame; otherwise raises `TypeError`. |
## Test plan
7 tests in `tests/expr/test_dataframe_cross_join.py`:
- **Positive**: 2×2; multi-column on both sides; 1×N; empty×N (zero rows); aliased sides sharing a key name (row-count check, since `to_pandas` collapses qualifiers on duplicate names).
- **Lazy return**: `isinstance(out, DataFrame)`.
- **Error**: non-DataFrame `other` → `TypeError`.
Output assertions use exact `pd.testing.assert_frame_equal` after sorting where row order isn't unique.
Local: 7 unit + 25 doctests + `ruff check` + `cargo fmt --check` all clean. Re-ran the 21 join tests from #908 — no regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #925:
URL: https://github.com/apache/sedona-db/pull/925#discussion_r3375526758
##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -796,6 +796,43 @@ def join(
return DataFrame(self._ctx, joined_impl.select(projection))
+ def cross_join(self, other: "DataFrame") -> "DataFrame":
+ """Cartesian product of two DataFrames.
+
+ Returns a DataFrame containing every pair of rows from `self`
+ and `other`. The result row count is `len(self) * len(other)`.
+ Both sides' columns are kept verbatim — disambiguate with
+ `df.alias(...)` on either side if column names collide.
Review Comment:
The docstring claims the result row count is `len(self) * len(other)`, but `DataFrame.__len__` raises for lazy frames. This is user-facing and misleading; rephrase without suggesting `len(...)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #925:
URL: https://github.com/apache/sedona-db/pull/925#discussion_r3378150172
##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -796,6 +796,43 @@ def join(
return DataFrame(self._ctx, joined_impl.select(projection))
+ def cross_join(self, other: "DataFrame") -> "DataFrame":
+ """Cartesian product of two DataFrames.
+
+ Returns a DataFrame containing every pair of rows from `self`
+ and `other`. The result row count is `len(self) * len(other)`.
+ Both sides' columns are kept verbatim — disambiguate with
+ `df.alias(...)` on either side if column names collide.
Review Comment:
Good catch — fixed in d748394. The docstring now reads:
> Returns a DataFrame containing every pair of rows from `self` and `other`; the row count is the product of the two input row counts.
No `len(...)` reference, so no misleading suggestion of an API that would actually raise on a lazy frame.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu merged PR #925:
URL: https://github.com/apache/sedona-db/pull/925
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #925:
URL: https://github.com/apache/sedona-db/pull/925#discussion_r3381190382
##########
python/sedonadb/tests/expr/test_dataframe_cross_join.py:
##########
@@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pandas as pd
+import pandas.testing as pdt
+import pytest
+
+from sedonadb.dataframe import DataFrame
+
+
+def test_cross_join_basic(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2]}))
+ right = con.create_data_frame(pd.DataFrame({"y": ["a", "b"]}))
+ out = left.cross_join(right).sort("x", "y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame({"x": [1, 1, 2, 2], "y": ["a", "b", "a", "b"]}),
+ )
+
+
+def test_cross_join_multi_column_sides(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2], "v": ["p", "q"]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [10, 20], "w": ["r", "s"]}))
+ out = left.cross_join(right).sort("x", "y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame(
+ {
+ "x": [1, 1, 2, 2],
+ "v": ["p", "p", "q", "q"],
+ "y": [10, 20, 10, 20],
+ "w": ["r", "s", "r", "s"],
+ }
+ ),
+ )
+
+
+def test_cross_join_single_row_left(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [10, 20, 30]}))
+ out = left.cross_join(right).sort("y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame({"x": [1, 1, 1], "y": [10, 20, 30]}),
+ )
+
+
+def test_cross_join_empty_right(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2]}))
+ right = con.create_data_frame(pd.DataFrame({"y": pd.Series([], dtype="int64")}))
+ out = left.cross_join(right).to_pandas()
+ assert len(out) == 0
+ assert list(out.columns) == ["x", "y"]
+
+
+def test_cross_join_with_aliases_disambiguates_names(con):
+ # When both sides share a column name, aliasing keeps the output
+ # rows but qualifiers vanish in to_pandas() — so just verify the
+ # join builds and produces the expected row count.
+ left = con.create_data_frame(pd.DataFrame({"k": [1, 2]})).alias("l")
+ right = con.create_data_frame(pd.DataFrame({"k": [10, 20, 30]})).alias("r")
+ out = left.cross_join(right)
+ assert out.count() == 6
+
+
+def test_cross_join_returns_lazy_dataframe(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [1]}))
+ assert isinstance(left.cross_join(right), DataFrame)
Review Comment:
Unless there's bindings surface you're testing with these, probably just the two tests will do. They can live in the other join file I think since there's only two.
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(python/sedonadb): add DataFrame.cross_join [sedona-db]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #925:
URL: https://github.com/apache/sedona-db/pull/925#discussion_r3383154072
##########
python/sedonadb/tests/expr/test_dataframe_cross_join.py:
##########
@@ -0,0 +1,89 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pandas as pd
+import pandas.testing as pdt
+import pytest
+
+from sedonadb.dataframe import DataFrame
+
+
+def test_cross_join_basic(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2]}))
+ right = con.create_data_frame(pd.DataFrame({"y": ["a", "b"]}))
+ out = left.cross_join(right).sort("x", "y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame({"x": [1, 1, 2, 2], "y": ["a", "b", "a", "b"]}),
+ )
+
+
+def test_cross_join_multi_column_sides(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2], "v": ["p", "q"]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [10, 20], "w": ["r", "s"]}))
+ out = left.cross_join(right).sort("x", "y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame(
+ {
+ "x": [1, 1, 2, 2],
+ "v": ["p", "p", "q", "q"],
+ "y": [10, 20, 10, 20],
+ "w": ["r", "s", "r", "s"],
+ }
+ ),
+ )
+
+
+def test_cross_join_single_row_left(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [10, 20, 30]}))
+ out = left.cross_join(right).sort("y").to_pandas()
+ pdt.assert_frame_equal(
+ out,
+ pd.DataFrame({"x": [1, 1, 1], "y": [10, 20, 30]}),
+ )
+
+
+def test_cross_join_empty_right(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1, 2]}))
+ right = con.create_data_frame(pd.DataFrame({"y": pd.Series([], dtype="int64")}))
+ out = left.cross_join(right).to_pandas()
+ assert len(out) == 0
+ assert list(out.columns) == ["x", "y"]
+
+
+def test_cross_join_with_aliases_disambiguates_names(con):
+ # When both sides share a column name, aliasing keeps the output
+ # rows but qualifiers vanish in to_pandas() — so just verify the
+ # join builds and produces the expected row count.
+ left = con.create_data_frame(pd.DataFrame({"k": [1, 2]})).alias("l")
+ right = con.create_data_frame(pd.DataFrame({"k": [10, 20, 30]})).alias("r")
+ out = left.cross_join(right)
+ assert out.count() == 6
+
+
+def test_cross_join_returns_lazy_dataframe(con):
+ left = con.create_data_frame(pd.DataFrame({"x": [1]}))
+ right = con.create_data_frame(pd.DataFrame({"y": [1]}))
+ assert isinstance(left.cross_join(right), DataFrame)
Review Comment:
Done in 0e0e5e9 — trimmed to the two tests (`test_cross_join_basic`, `test_cross_join_non_dataframe_other_raises`) and folded them into `test_dataframe_join.py`. Deleted the dedicated `test_dataframe_cross_join.py` file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org