[jira] [Updated] (SPARK-40005) Self-contained examples with parameter descriptions in PySpark documentation

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-40005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-40005:
---------------------------------
    Target Version/s: 3.4.0

> Self-contained examples with parameter descriptions in PySpark documentation
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-40005
>                 URL: https://issues.apache.org/jira/browse/SPARK-40005
>             Project: Spark
>          Issue Type: Umbrella
>          Components: Documentation, PySpark
>    Affects Versions: 3.4.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>
> This JIRA aims to improve PySpark documentation in:
> - {{pyspark}}
> - {{pyspark.ml}}
> - {{pyspark.sql}}
> - {{pyspark.sql.streaming}}
> We should:
> - Make the examples self-contained, e.g., https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot.html
> - Document {{Parameters}} https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot.html#pandas.DataFrame.pivot. There are many API that misses parameters in PySpark, e.g., [DataFrame.union|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.union.html#pyspark.sql.DataFrame.union]
> If the size of file is large, e.g., dataframe.py, we should split that down into each subtask, and improve documentation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org