You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/22 01:58:21 UTC
[GitHub] [spark] Yikun edited a comment on pull request #32276: [SPARK-35173][PYTHON][SQL] Add multiple columns adding support for PySpark
Yikun edited a comment on pull request #32276:
URL: https://github.com/apache/spark/pull/32276#issuecomment-824472080
> As you already noticed these methods are private and, as such, not intended for end-user.
~@zero323 Maybe I bring some misunderstanding in previous , I didn't expose the **private withColumns** API in PySpark, just to match the ability of the scala withColumn API. That means, the scala [withColumn API](https://github.com/apache/spark/blob/b5241c97b17a1139a4ff719bfce7f68aef094d95/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2396) can receive multiple columns now, and support by calling the [internal withColumns API](https://github.com/apache/spark/blob/b5241c97b17a1139a4ff719bfce7f68aef094d95/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2396-L2402).~
> we should think about having an API in Scala too to match. It was sort of rejected once at SPARK-12225
@HyukjinKwon ~See the first reply for misunderstanding, actually, SPARK-12225 have already support the multiple columns adding support on scala withColumn API in https://github.com/apache/spark/commit/3ca367083e196e6487207211e6c49d4bbfe31288 ([SPARK-22001](https://issues.apache.org/jira/browse/SPARK-22001))~ Yes, we only add the multiple columns adding support on scala private withColumns in https://github.com/apache/spark/commit/3ca367083e196e6487207211e6c49d4bbfe31288 ([SPARK-22001](https://issues.apache.org/jira/browse/SPARK-22001))
> Additionally, naming would be rather confusing.
> we should discuss adding such functionality for all APIs and having less confusing name.
Sure, I will send an email to dev ML to decribe these confusing naming things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org