You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "jiayuasu (via GitHub)" <gi...@apache.org> on 2026/05/15 06:45:46 UTC

[PR] feat(python/sedonadb): add DataFrame __getitem__ [sedona-db]

jiayuasu opened a new pull request, #846:
URL: https://github.com/apache/sedona-db/pull/846

   Pandas-style bracket indexing on `DataFrame`, dispatching to the existing `select` / `filter` / `col` paths landed in earlier PRs (#807, #823, #832, #835).
   
   This is a small follow-on PR continuing Phase P1/P2 of #791. Pure Python sugar — no new Rust code.
   
   ## What's new
   
   ```python
   df["x"]                       # → Expr(col("x"))
   df[["x", "y"]]                # → DataFrame.select("x", "y")
   df[df["x"] > 0]               # → DataFrame.filter(col("x") > 0)
   df[(df["x"] + df["y"]) > 30]  # composes cleanly with operator overloads
   df[df["x"] > 1][["y"]]        # chain: filter → project
   ```
   
   Row-position indexing (ints, slices, `.loc`, `.iloc`) is intentionally **not** supported and raises `TypeError` with a message pointing at the three supported forms. SedonaDB has no row-ordering or index concept in this scope; the non-goal is documented in [#791](https://github.com/apache/sedona-db/issues/791).
   
   ## Note on `df["x"]` return type
   
   For now, `df["x"]` returns an `Expr` (a column reference). A `Series` wrapper is planned for the next phase — when that lands, `df["x"]` will start returning `Series` instead and `Expr` will fall back to an internal escape hatch. The compose-with-operators idiom continues to work either way because `Series` will share the same operator surface as `Expr`.
   
   ## Test plan
   
   10 tests in `tests/expr/test_dataframe_getitem.py`:
   
   - **Positive**: string lookup → Expr (exact `repr()`); list projection (two-col, single-col, reorder); bool-Expr filter; arithmetic compose; filter-then-project chain.
   - **Errors**: list with non-string element → `TypeError`; int → `TypeError`; slice → `TypeError`.
   
   All 10 pass locally. No regressions in existing tests; doctests + `ruff format` + `ruff check` all clean.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(python/sedonadb): add DataFrame __getitem__ [sedona-db]

Posted by "zhangfengcdt (via GitHub)" <gi...@apache.org>.
zhangfengcdt merged PR #846:
URL: https://github.com/apache/sedona-db/pull/846


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(python/sedonadb): add DataFrame __getitem__ [sedona-db]

Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on PR #846:
URL: https://github.com/apache/sedona-db/pull/846#issuecomment-4469654066

   @paleolimbot I’ll create a follow-up PR for this. I’d still like to keep the round-robin review assignment so our engineers can build familiarity with different components of the project. That said, this does not mean I’ll merge a PR solely based on their approval.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(python/sedonadb): add DataFrame __getitem__ [sedona-db]

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #846:
URL: https://github.com/apache/sedona-db/pull/846#discussion_r3249600175


##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -88,6 +88,57 @@ def head(self, n: int = 5) -> "DataFrame":
         """
         return self.limit(n)
 
+    def __getitem__(self, key):
+        """Index into the DataFrame using pandas-style bracket access.
+
+        Three forms are supported:
+
+        - `df["x"]` returns an `Expr` referencing column `x`. Equivalent to
+          `sedonadb.expr.col("x")`. Note that this returns an `Expr`, not a
+          materialized column — the `Series` type that pandas users
+          eventually expect will land in a future phase.
+        - `df[["x", "y"]]` returns a new `DataFrame` with the listed columns
+          (equivalent to `df.select("x", "y")`).
+        - `df[bool_expr]` returns a new `DataFrame` filtered by the boolean
+          expression (equivalent to `df.filter(bool_expr)`).
+
+        Row-position indexing (integers, slices, `.loc`, `.iloc`) is
+        intentionally not supported — SedonaDB has no row ordering or
+        index concept in this scope.
+
+        Examples:
+
+            >>> from sedonadb.expr import col
+            >>> sd = sedona.db.connect()
+            >>> df = sd.sql("SELECT * FROM (VALUES (1, 10), (2, 20), (3, 30)) AS t(x, y)")
+            >>> df["x"]
+            Expr(x)
+            >>> df[["x", "y"]].count()
+            3
+            >>> df[df["x"] > 1].count()
+            2
+        """
+        from sedonadb.expr import Expr
+        from sedonadb.expr import col as _col

Review Comment:
   These should be module-level imports (and they should be combined). Lazy imports in this module are only for optional dependencies (pyarrow is the usual culprit)



##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -88,6 +88,57 @@ def head(self, n: int = 5) -> "DataFrame":
         """
         return self.limit(n)
 
+    def __getitem__(self, key):

Review Comment:
   ```suggestion
       def __getitem__(self, key: Union[str, int]) -> Expr:
   ```
   
   Eventually we can give `Expr` a subclass for columns to improve the type hinting



##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -88,6 +88,57 @@ def head(self, n: int = 5) -> "DataFrame":
         """
         return self.limit(n)
 
+    def __getitem__(self, key):
+        """Index into the DataFrame using pandas-style bracket access.
+
+        Three forms are supported:
+
+        - `df["x"]` returns an `Expr` referencing column `x`. Equivalent to
+          `sedonadb.expr.col("x")`. Note that this returns an `Expr`, not a
+          materialized column — the `Series` type that pandas users
+          eventually expect will land in a future phase.
+        - `df[["x", "y"]]` returns a new `DataFrame` with the listed columns
+          (equivalent to `df.select("x", "y")`).
+        - `df[bool_expr]` returns a new `DataFrame` filtered by the boolean
+          expression (equivalent to `df.filter(bool_expr)`).
+
+        Row-position indexing (integers, slices, `.loc`, `.iloc`) is
+        intentionally not supported — SedonaDB has no row ordering or
+        index concept in this scope.
+
+        Examples:
+
+            >>> from sedonadb.expr import col
+            >>> sd = sedona.db.connect()
+            >>> df = sd.sql("SELECT * FROM (VALUES (1, 10), (2, 20), (3, 30)) AS t(x, y)")
+            >>> df["x"]
+            Expr(x)
+            >>> df[["x", "y"]].count()
+            3
+            >>> df[df["x"] > 1].count()
+            2
+        """
+        from sedonadb.expr import Expr
+        from sedonadb.expr import col as _col
+
+        if isinstance(key, str):
+            return _col(key)

Review Comment:
   This should reach into the underlying `DFSchema` and pull out the actual qualified column expression (displaying a nice error for columns that don't exist). You'll need this for join expressions.



##########
python/sedonadb/python/sedonadb/dataframe.py:
##########
@@ -88,6 +88,57 @@ def head(self, n: int = 5) -> "DataFrame":
         """
         return self.limit(n)
 
+    def __getitem__(self, key):
+        """Index into the DataFrame using pandas-style bracket access.
+
+        Three forms are supported:
+
+        - `df["x"]` returns an `Expr` referencing column `x`. Equivalent to
+          `sedonadb.expr.col("x")`. Note that this returns an `Expr`, not a
+          materialized column — the `Series` type that pandas users
+          eventually expect will land in a future phase.
+        - `df[["x", "y"]]` returns a new `DataFrame` with the listed columns
+          (equivalent to `df.select("x", "y")`).
+        - `df[bool_expr]` returns a new `DataFrame` filtered by the boolean
+          expression (equivalent to `df.filter(bool_expr)`).
+
+        Row-position indexing (integers, slices, `.loc`, `.iloc`) is
+        intentionally not supported — SedonaDB has no row ordering or
+        index concept in this scope.
+
+        Examples:
+
+            >>> from sedonadb.expr import col
+            >>> sd = sedona.db.connect()
+            >>> df = sd.sql("SELECT * FROM (VALUES (1, 10), (2, 20), (3, 30)) AS t(x, y)")
+            >>> df["x"]
+            Expr(x)
+            >>> df[["x", "y"]].count()
+            3
+            >>> df[df["x"] > 1].count()
+            2
+        """
+        from sedonadb.expr import Expr
+        from sedonadb.expr import col as _col
+
+        if isinstance(key, str):
+            return _col(key)
+        if isinstance(key, Expr):
+            return self.filter(key)
+        if isinstance(key, list):
+            for k in key:
+                if not isinstance(k, str):
+                    raise TypeError(
+                        f"DataFrame[list] expects a list of column names, "
+                        f"got {type(k).__name__}"
+                    )
+            return self.select(*key)
+        raise TypeError(
+            f"DataFrame indexing is not supported for {type(key).__name__}. "
+            f"Use df['x'] for a column expression, df[['x', 'y']] to project "
+            f"columns, or df[bool_expr] to filter rows."
+        )

Review Comment:
   I think we should restrict this to only the first case. The ability to put a filter or multi-column selection here is pandas-y but prevents type hinting from working well on the output (i.e., IDEs and LLMs that use the language server or can use type hints can't auto complete `tab["col"].<tab>`).
   
   This should also support integer indexing (e.g., `tab[0]` gives you the first column).
   
   FWIW Ibis used to accept the second two cases but deprecated them (`FutureWarning: Selecting/filtering arbitrary expressions in `Table.__getitem__` is deprecated and will be removed in version 10.0. Please use `Table.select` or `Table.filter` instead.`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org