[GitHub] [spark] yaroslav-serhiichuk commented on pull request #31768: [SPARK-33436][PYSPARK] PySpark equivalent of SparkContext.hadoopConfiguration

[GitBox <gi...@apache.org>]

yaroslav-serhiichuk commented on pull request #31768:
URL: https://github.com/apache/spark/pull/31768#issuecomment-792345441


   I had faced this issue when tried to setup s3a properties in pyspark job. The way how to do it was not present anywhere in the documentation but found a solution on  [Stackoverflow](https://stackoverflow.com/questions/28844631/how-to-set-hadoop-configuration-values-from-pyspark). Digging deeper I found a similar ticket opened in [Jira](https://issues.apache.org/jira/browse/SPARK-33436). That's why this PR was created. If we really do not need it, it can be closed. But in my mind, it is logical to have the same sc API I Scala and python.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org