You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/08/23 08:54:05 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request, #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
HyukjinKwon opened a new pull request, #42628:
URL: https://github.com/apache/spark/pull/42628
### What changes were proposed in this pull request?
This PR proposes the alias name `sf` instead of `F` for `pyspark.sql.functions` alias in public documentation:
```python
from pyspark.sql import functions as sf
```
This PR does not change the internal or test codes as it's too invasive, and might easily cause conflicts.
### Why are the changes needed?
```python
from pyspark.sql import functions as F
```
isn’t very Pythonic - it does not follow PEP 8, see [Package and Module Names](https://peps.python.org/pep-0008/#package-and-module-names).
> Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves
> readability. Python packages should also have short, all-lowercase names, although the use of underscores
> is discouraged.
Therefore, the module’s alias should follow this. In practice, the uppercase is only used at the module/package
level constants in my experience, see also [Constants](https://peps.python.org/pep-0008/#constants).
See also [this stackoverflow comment](https://stackoverflow.com/questions/70458086/how-to-correctly-import-pyspark-sql-functions#comment129714058_70458115).
### Does this PR introduce _any_ user-facing change?
Yes, it changes documentation so users
### How was this patch tested?
Manually checked.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42628:
URL: https://github.com/apache/spark/pull/42628#discussion_r1302719588
##########
python/pyspark/sql/functions.py:
##########
@@ -11572,11 +11572,11 @@ def explode(col: "ColumnOrName") -> Column:
Examples
--------
>>> from pyspark.sql import Row
- >>> eDF = spark.createDataFrame([Row(a=1, intlist=[1,2,3], mapfield={"a": "b"})])
- >>> eDF.select(explode(eDF.intlist).alias("anInt")).collect()
+ >>> edf = spark.createDataFrame([Row(a=1, intlist=[1,2,3], mapfield={"a": "b"})])
Review Comment:
nit: I guess we should use `df` here, `edf` or `eDF` seems weird
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1689579294
Will parts of the code be completed by a follow-up, or will it not be fixed for now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1689589433
> I think we should leave them for now. I initially changed all .. then realised that there are too many files being touched.. so I changed them back. That will cause conflicts too much when we backport, etc. Let's leave them for now.
OK ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1689586924
I think we should leave them for now. I initially changed all .. then realised that there are too many files being touched.. so I changed them back. That will cause conflicts too much when we backport, etc. Let's leave them for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1689557620
cc @ueshin and @itholic too FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
URL: https://github.com/apache/spark/pull/42628
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1689556034
cc @zhengruifeng @LuciferYang @allisonwang-db
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #42628: [SPARK-44928][PYTHON][DOCS] Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #42628:
URL: https://github.com/apache/spark/pull/42628#issuecomment-1690943611
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org