[spark] branch branch-3.1 updated: [SPARK-37288][PYTHON][3.2] Backport since annotation update

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.1 by this push:
     new e397c77  [SPARK-37288][PYTHON][3.2] Backport since annotation update
e397c77 is described below

commit e397c77a08ab27960ff9594bed88b9f16cc3f628
Author: zero323 <ms...@gmail.com>
AuthorDate: Sat Nov 13 23:53:24 2021 +0900

    [SPARK-37288][PYTHON][3.2] Backport since annotation update
    
    ### What changes were proposed in this pull request?
    
    This PR changes since annotation to support `float` arguments:
    
    ```python
    def since(version: Union[str, float]) -> Callable[[T], T]: ...
    ```
    
    ### Why are the changes needed?
    
    `since` is used with both `str` and `float` both in Spark and related libraries and this change has been already done for Spark >= 3.3 (SPARK-36906),
    
    Note that this technically fixes a bug in the downstream projects that run mypy checks against `pyspark.since`. When they use it, for example, with `pyspark.since(3.2)`, mypy checks fails; however, this case is legitimate. After this change, the mypy check can pass in thier CIs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    ```python
    since(3.2)
    def f():
        ...
    ```
    
    is going to type check if downstream projects run mypy to validate the types.
    
    Otherwise, it does not affect anything invasive or user-facing behavior change.
    
    ### How was this patch tested?
    
    Existing tests and manual testing.
    
    Closes #34555 from zero323/SPARK-37288.
    
    Authored-by: zero323 <ms...@gmail.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
    (cherry picked from commit bdde69363f75f4f37370e38cddb47cd3631168b0)
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/__init__.pyi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/python/pyspark/__init__.pyi b/python/pyspark/__init__.pyi
index 852de35..8f5058c 100644
--- a/python/pyspark/__init__.pyi
+++ b/python/pyspark/__init__.pyi
@@ -16,7 +16,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
-from typing import Callable, Optional, TypeVar
+from typing import Callable, Optional, TypeVar, Union
 
 from pyspark.accumulators import (  # noqa: F401
     Accumulator as Accumulator,
@@ -64,7 +64,7 @@ from pyspark.sql import (  # noqa: F401
 T = TypeVar("T")
 F = TypeVar("F", bound=Callable)
 
-def since(version: str) -> Callable[[T], T]: ...
+def since(version: Union[str, float]) -> Callable[[T], T]: ...
 def copy_func(
     f: F,
     name: Optional[str] = ...,

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org