[jira] [Created] (SPARK-36617) Inconsistencies in approxQuantile annotations

["Maciej Szymkiewicz (Jira)" <ji...@apache.org>]

Maciej Szymkiewicz created SPARK-36617:
------------------------------------------

             Summary: Inconsistencies in approxQuantile annotations
                 Key: SPARK-36617
                 URL: https://issues.apache.org/jira/browse/SPARK-36617
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, SQL
    Affects Versions: 3.1.0, 3.2.0, 3.3.0
            Reporter: Maciej Szymkiewicz


I've been reviewing PR in the legacy repo (https://github.com/zero323/pyspark-stubs/pull/552) and it looks like we have two problems with annotations for {{approxQuantile}}.

First of all {{DataFrame.approxQuantile}} should overload definition to match input arguments ‒ if col is a sequence then result should be a list of lists:

{code:python}
    @overload
    def approxQuantile(
        self,
        col: str,
        probabilities: Union[List[float], Tuple[float]],
        relativeError: float
    ) -> List[float]: ...
    @overload
    def approxQuantile(
        self,
        col: Union[List[str], Tuple[str]],
        probabilities: Union[List[float], Tuple[float]],
        relativeError: float
    ) -> List[List[float]]: ...
{code}


Additionally {{DataFrameStatFunctions.approxQuantile}} should match whatever we have in {{DataFrame}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org