You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Maciej Szymkiewicz (Jira)" <ji...@apache.org> on 2021/10/15 20:46:00 UTC
[jira] [Comment Edited] (SPARK-37022) Use black as a formatter for
the whole PySpark codebase.
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429477#comment-17429477 ]
Maciej Szymkiewicz edited comment on SPARK-37022 at 10/15/21, 8:45 PM:
-----------------------------------------------------------------------
The attached files show git diff stats for a given configuration.
I'll open a draft PR soon, to better visualize the extent of required changes (https://github.com/apache/spark/pull/34297)
was (Author: zero323):
The attached files show git diff stats for a given configuration.
I'll open a draft PR soon, to better visualize the extent of required changes.
> Use black as a formatter for the whole PySpark codebase.
> --------------------------------------------------------
>
> Key: SPARK-37022
> URL: https://issues.apache.org/jira/browse/SPARK-37022
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Maciej Szymkiewicz
> Priority: Major
> Attachments: black-diff-stats.txt, pyproject.toml
>
>
> [{{black}}|https://github.com/psf/black] is a popular Python code formatter. It is used by a number of projects, both small and large, including prominent ones, like pandas, scikit-learn, Django or SQLAlchemy. Black is already used to format a {{pyspark.pandas}} and (though not enforced) stubs files.
> We should consider using black to enforce formatting of all PySpark files. There are multiple reasons to do that:
> - Consistency: black is already used across existing codebase and black formatted chunks of code are already added to modules other than pyspark.pandas as a result of type hints inlining (SPARK-36845).
> - Lower cost of contributing and reviewing: Formatting can be automatically enforced and applied.
> - Simplify reviews: In general, black formatted code, produces small and highly readable diffs.
> Risks:
> - Initial reformatting requires quite significant changes.
> - Applying black will break blame in GitHub UI (for git in general see [Avoiding ruining git blame|https://black.readthedocs.io/en/stable/guides/introducing_black_to_your_project.html?highlight=blame#avoiding-ruining-git-blame]).
> Additional steps:
> - To simplify backporting, black will have to be applied to all active branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org