You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2026/03/23 03:01:07 UTC
[PR] [SPARK-56135][DOCS] Add option to use venv in PySpark docs [spark]
Yicong-Huang opened a new pull request, #54943:
URL: https://github.com/apache/spark/pull/54943
### What changes were proposed in this pull request?
Add a `venv` section in the PySpark documentation as an alternative to Conda for environment management:
- `python/docs/source/getting_started/install.rst` - new "Using venv" section alongside "Using Conda"
- `python/docs/source/development/contributing.rst` - new "venv" section alongside Conda and pip under Environment Setup
### Why are the changes needed?
The PySpark docs currently only mention pip (without isolation) and Conda for environment setup. Python's built-in `venv` module is a lightweight alternative that doesn't require installing Conda. Adding this option gives users more choices for setting up their development environment.
### Does this PR introduce _any_ user-facing change?
Documentation only.
### How was this patch tested?
N/A (documentation change).
### Was this patch authored or co-authored using generative AI tooling?
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56135][DOCS] Add option to use venv in PySpark docs [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #54943:
URL: https://github.com/apache/spark/pull/54943#issuecomment-4108865050
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56135][DOCS] Add option to use venv in PySpark docs [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #54943:
URL: https://github.com/apache/spark/pull/54943#discussion_r2973002545
##########
python/docs/source/development/contributing.rst:
##########
@@ -142,6 +142,18 @@ Once it is set up, make sure you switch to `pyspark-dev-env` before starting the
Now, you can start developing and `running the tests <testing.rst>`_.
+venv
+~~~~
+
+You can use Python's built-in ``venv`` module to create an isolated environment:
+
+.. code-block:: bash
+
+ # Python 3.10+ is required
+ python3 -m venv .venv
+ source .venv/bin/activate # On Windows: .venv\Scripts\activate
Review Comment:
can we not mention windows?
since we even don't have tests run on windows.
this comment suggest pyspark is well supported on windows, but I guess it is not true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56135][DOCS] Add option to use venv in PySpark docs [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #54943: [SPARK-56135][DOCS] Add option to use venv in PySpark docs
URL: https://github.com/apache/spark/pull/54943
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org