[spark] branch branch-3.1 updated: [SPARK-34630][PYTHON] Add typehint for pyspark.__version__

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.1 by this push:
     new b5adfaa  [SPARK-34630][PYTHON] Add typehint for pyspark.__version__
b5adfaa is described below

commit b5adfaaa8057a9c376efb1e99fb2471a4bfc4680
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Sun Apr 11 10:40:08 2021 +0900

    [SPARK-34630][PYTHON] Add typehint for pyspark.__version__
    
    ### What changes were proposed in this pull request?
    This PR adds the typehint of pyspark.__version__, which was mentioned in [SPARK-34630](https://issues.apache.org/jira/browse/SPARK-34630).
    
    ### Why are the changes needed?
    There were some short discussion happened in https://github.com/apache/spark/pull/31823#discussion_r593830911 .
    
    After further deep investigation on [1][2], we can see the `pyspark.__version__` is added by [setup.py](https://github.com/apache/spark/blob/c06758834e6192b1888b4a885c612a151588b390/python/setup.py#L201), it makes `__version__` embedded into pyspark module, that means the `__init__.pyi` is the right place to add the typehint for `__version__`.
    
    So, this patch adds the type hint `__version__` in pyspark/__init__.pyi.
    
    [1] [PEP-396 Module Version Numbers](https://www.python.org/dev/peps/pep-0396/)
    [2] https://packaging.python.org/guides/single-sourcing-package-version/
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    1. Disable the ignore_error on
    https://github.com/apache/spark/blob/ee7bf7d9628384548d09ac20e30992dd705c20f4/python/mypy.ini#L132
    
    2. Run mypy:
    - Before fix
    ```shell
    (venv) ➜  spark git:(SPARK-34629) ✗ mypy --config-file python/mypy.ini python/pyspark | grep version
    python/pyspark/pandas/spark/accessors.py:884: error: Module has no attribute "__version__"
    ```
    
    - After fix
    ```shell
    (venv) ➜  spark git:(SPARK-34629) ✗ mypy --config-file python/mypy.ini python/pyspark | grep version
    ```
    no output
    
    Closes #32110 from Yikun/SPARK-34629.
    
    Authored-by: Yikun Jiang <yi...@gmail.com>
    Signed-off-by: HyukjinKwon <gu...@apache.org>
    (cherry picked from commit 4c1ccdabe8d5c0012ce481822c07beaf8e7ead0c)
    Signed-off-by: HyukjinKwon <gu...@apache.org>
---
 python/pyspark/__init__.pyi | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/python/pyspark/__init__.pyi b/python/pyspark/__init__.pyi
index ef07c32..cf4bb02 100644
--- a/python/pyspark/__init__.pyi
+++ b/python/pyspark/__init__.pyi
@@ -71,3 +71,5 @@ def copy_func(
     doc: Optional[str] = ...,
 ) -> F: ...
 def keyword_only(func: F) -> F: ...
+
+__version__: str

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org