You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "ntjohnson1 (via GitHub)" <gi...@apache.org> on 2026/03/04 20:19:53 UTC

[PR] Enable doc tests in local and CI testing [datafusion-python]

ntjohnson1 opened a new pull request, #1409:
URL: https://github.com/apache/datafusion-python/pull/1409

   # Which issue does this PR close?
   
   Part 1 of https://github.com/apache/datafusion-python/pull/1400
   Related to https://github.com/apache/datafusion-python/issues/1394
   
    # Rationale for this change
   This enables doctests so if more examples are added to doc strings they can be validated locally and in CI.
   
   # What changes are included in this PR?
   * Turns on doctest so pytest verifies docstring examples
   * Minor update to the test.yml to fix test discovery with doctest turned on, and fixed a github lint that triggered for me since I touched that file
   * Fixes a few existing examples to be compliant
   
   # Are there any user-facing changes?
   No


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Enable doc tests in local and CI testing [datafusion-python]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #1409:
URL: https://github.com/apache/datafusion-python/pull/1409#discussion_r2886415731


##########
python/datafusion/dataframe.py:
##########
@@ -1389,9 +1390,12 @@ def fill_null(self, value: Any, subset: list[str] | None = None) -> DataFrame:
             DataFrame with null values replaced where type casting is possible
 
         Examples:
-            >>> df = df.fill_null(0)  # Fill all nulls with 0 where possible
-            >>> # Fill nulls in specific string columns
-            >>> df = df.fill_null("missing", subset=["name", "category"])
+            >>> from datafusion import SessionContext, col

Review Comment:
   Thanks @ntjohnson1 
   
   Did you verify from the CI run logs that these tests are now run? I took a brief look and couldn't find anything that was new https://github.com/apache/datafusion-python/actions/runs/22687531822/job/65776433980?pr=1409 
   
   🤔 



##########
pyproject.toml:
##########
@@ -70,6 +70,9 @@ features = ["substrait"]
 [tool.pytest.ini_options]
 asyncio_mode = "auto"
 asyncio_default_fixture_loop_scope = "function"
+addopts = "--doctest-modules"

Review Comment:
   matches https://docs.pytest.org/en/stable/how-to/doctest.html 👍 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Enable doc tests in local and CI testing [datafusion-python]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #1409:
URL: https://github.com/apache/datafusion-python/pull/1409#issuecomment-4005978736

   I am going to say let's get this party started!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Enable doc tests in local and CI testing [datafusion-python]

Posted by "ntjohnson1 (via GitHub)" <gi...@apache.org>.
ntjohnson1 commented on code in PR #1409:
URL: https://github.com/apache/datafusion-python/pull/1409#discussion_r2886473499


##########
python/datafusion/dataframe.py:
##########
@@ -1389,9 +1390,12 @@ def fill_null(self, value: Any, subset: list[str] | None = None) -> DataFrame:
             DataFrame with null values replaced where type casting is possible
 
         Examples:
-            >>> df = df.fill_null(0)  # Fill all nulls with 0 where possible
-            >>> # Fill nulls in specific string columns
-            >>> df = df.fill_null("missing", subset=["name", "category"])
+            >>> from datafusion import SessionContext, col

Review Comment:
   The doctests run as a part of pytest so if you click on `Run Tests` you can see the new doctest entries.
   
   There aren't that many examples in the code at the moment. 
   If you scroll all the way to the bottom you can now see source code showing up in addition to test code
   ```console
   python/datafusion/dataframe.py::datafusion.dataframe.DataFrame.fill_null PASSED [ 99%]
   python/datafusion/dataframe.py::datafusion.dataframe.DataFrame.into_view PASSED [ 99%]
   python/datafusion/dataframe_formatter.py::datafusion.dataframe_formatter.configure_formatter PASSED [ 99%]
   python/datafusion/dataframe_formatter.py::datafusion.dataframe_formatter.get_formatter PASSED [ 99%]
   python/datafusion/dataframe_formatter.py::datafusion.dataframe_formatter.reset_formatter PASSED [ 99%]
   python/datafusion/dataframe_formatter.py::datafusion.dataframe_formatter.set_formatter PASSED [100%]
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Enable doc tests in local and CI testing [datafusion-python]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #1409:
URL: https://github.com/apache/datafusion-python/pull/1409#discussion_r2886544405


##########
python/datafusion/dataframe.py:
##########
@@ -1389,9 +1390,12 @@ def fill_null(self, value: Any, subset: list[str] | None = None) -> DataFrame:
             DataFrame with null values replaced where type casting is possible
 
         Examples:
-            >>> df = df.fill_null(0)  # Fill all nulls with 0 where possible
-            >>> # Fill nulls in specific string columns
-            >>> df = df.fill_null("missing", subset=["name", "category"])
+            >>> from datafusion import SessionContext, col

Review Comment:
   Nice -- I checked https://github.com/apache/datafusion-python/actions/runs/22676065624/job/65741468321 (a recent run from main) and indeed those tests are not there
   
   ```
   
   python/tests/test_view.py::test_register_filtered_dataframe PASSED       [ 99%]
   python/tests/test_wrapper_coverage.py::test_datafusion_missing_exports PASSED [100%]
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Enable doc tests in local and CI testing [datafusion-python]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #1409:
URL: https://github.com/apache/datafusion-python/pull/1409


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org