You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2022/07/05 11:34:00 UTC
[jira] [Created] (SPARK-39685) Linter build broken in branch-3.2 with Scala 2.13
Hyukjin Kwon created SPARK-39685:
------------------------------------
Summary: Linter build broken in branch-3.2 with Scala 2.13
Key: SPARK-39685
URL: https://issues.apache.org/jira/browse/SPARK-39685
Project: Spark
Issue Type: Test
Components: Build, PySpark, Tests
Affects Versions: 3.2.1
Reporter: Hyukjin Kwon
https://github.com/apache/spark/runs/7189971643?check_suite_focus=true
{code}
mypy checks failed:
python/pyspark/pandas/typedef/string_typehints.py:24: error: Name "np" already defined (by an import)
python/pyspark/pandas/typedef/string_typehints.py:30: error: Incompatible import of "DataFrame" (imported name has type "Type[pyspark.pandas.frame.DataFrame[Any]]", local name has type "Type[pandas.core.frame.DataFrame]")
python/pyspark/pandas/typedef/string_typehints.py:30: error: Incompatible import of "Series" (imported name has type "Type[pyspark.pandas.series.Series[Any]]", local name has type "Type[pandas.core.series.Series]")
python/pyspark/sql/pandas/_typing/__init__.pyi:39: error: Incompatible types in assignment (expression has type "Type[DataFrame]", variable has type "Type[DataFrameLike]")
python/pyspark/sql/pandas/_typing/__init__.pyi:40: error: Incompatible types in assignment (expression has type "Type[Series]", variable has type "Type[SeriesLike]")
python/pyspark/pandas/datetimes.py:37: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:39: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:52: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:67: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:74: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:81: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:88: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:95: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:102: error: Cannot determine type of "spark"
python/pyspark/pandas/datetimes.py:125: error: Cannot determine type of "spark"
python/pyspark/pandas/typedef/typehints.py:36: error: Module "pandas.api.types" has no attribute "pandas_dtype"
python/pyspark/pandas/typedef/typehints.py:526: error: Argument 3 to "DataFrameType" has incompatible type "List[None]"; expected "List[Optional[str]]"
python/pyspark/pandas/typedef/typehints.py:526: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
python/pyspark/pandas/typedef/typehints.py:526: note: Consider using "Sequence" instead, which is covariant
python/pyspark/pandas/utils.py:45: error: Module "pandas.api.types" has no attribute "is_list_like"
python/pyspark/pandas/internal.py:1540: error: Argument "name" to "StructField" has incompatible type "Optional[Hashable]"; expected "str"
python/pyspark/pandas/indexing.py:27: error: Module "pandas.api.types" has no attribute "is_list_like"
python/pyspark/pandas/indexing.py:993: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:994: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1014: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1021: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1023: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1024: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1055: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1067: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1079: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1139: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1142: error: Cannot determine type of "spark"
python/pyspark/pandas/indexing.py:1148: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:42: error: Module "pandas.api.types" has no attribute "is_list_like"
python/pyspark/pandas/generic.py:1206: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1207: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1293: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1294: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1379: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1380: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1452: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1453: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1520: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1521: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1590: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1648: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1790: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1791: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1871: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1872: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1986: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:1987: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2070: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2071: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2158: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2160: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2162: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2167: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:2168: error: Cannot determine type of "spark"
python/pyspark/pandas/generic.py:3213: error: Cannot determine type of "spark"
python/pyspark/pandas/frame.py:55: error: Module "pandas.api.types" has no attribute "is_list_like"
python/pyspark/pandas/frame.py:55: error: Module "pandas.api.types" has no attribute "is_dict_like"
python/pyspark/pandas/frame.py:55: error: Module "pandas.api.types" has no attribute "is_scalar"
python/pyspark/pandas/frame.py:65: error: Module "pandas.core.dtypes.common" has no attribute "_get_dtype_from_object"; maybe "infer_dtype_from_object"?
python/pyspark/pandas/frame.py:65: error: Name "infer_dtype_from_object" already defined (possibly by an import)
python/pyspark/pandas/frame.py:390: error: Incompatible types in assignment (expression has type "ExtensionDtype", variable has type "None")
python/pyspark/pandas/frame.py:679: error: Cannot determine type of "spark"
python/pyspark/pandas/frame.py:2525: error: Argument "axis" to "apply" of "DataFrame" has incompatible type "Union[int, str]"; expected "Union[Literal[0], Literal[1], Union[Literal['index'], Literal['columns']]]"
python/pyspark/pandas/frame.py:2539: error: Argument "axis" to "apply" of "DataFrame" has incompatible type "int"; expected "Union[Literal[0], Literal[1], Union[Literal['index'], Literal['columns']]]"
python/pyspark/pandas/frame.py:2760: error: Argument 2 to "transform" of "DataFrame" has incompatible type "int"; expected "Union[Literal[0], Literal[1], Union[Literal['index'], Literal['columns']]]"
python/pyspark/pandas/frame.py:4045: error: "Index" has no attribute "levels"; maybe "nlevels"?
python/pyspark/pandas/frame.py:4337: error: Cannot determine type of "spark"
python/pyspark/pandas/frame.py:5274: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:722: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:725: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:727: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:730: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:1432: error: Cannot determine type of "spark"
python/pyspark/pandas/window.py:1434: error: Cannot determine type of "spark"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "_all_kinds"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "BarPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "BoxPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "HistPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "MPLPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "PiePlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "AreaPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "LinePlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "BarhPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "ScatterPlot"
python/pyspark/pandas/plot/matplotlib.py:38: error: Module "pandas.plotting._core" has no attribute "KdePlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "BarPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "BoxPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "HistPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "PiePlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "AreaPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "LinePlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "BarhPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "ScatterPlot"
python/pyspark/pandas/plot/matplotlib.py:52: error: Module "pandas.plotting._matplotlib" has no attribute "KdePlot"
python/pyspark/pandas/plot/matplotlib.py:66: error: "Type[PlotAccessor]" has no attribute "_all_kinds"
python/pyspark/pandas/data_type_ops/num_ops.py:23: error: Module "pandas.api.types" has no attribute "is_bool_dtype"
python/pyspark/pandas/data_type_ops/num_ops.py:23: error: Module "pandas.api.types" has no attribute "is_integer_dtype"
python/pyspark/testing/pandasutils.py:26: error: Module "pandas.api.types" has no attribute "is_list_like"
python/pyspark/sql/tests/test_pandas_udf_typehints.py:180: error: Incompatible return value type (got "Union[Series, DataFrame]", expected "Series")
python/pyspark/sql/tests/test_pandas_udf_typehints.py:204: error: Incompatible return value type (got "floating[Any]", expected "float")
python/pyspark/pandas/tests/test_groupby.py:1992: error: Incompatible return value type (got "pandas.core.series.Series", expected "pyspark.pandas.series.Series[int]")
Found 339 errors in 31 files (checked 314 source files)
1
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org