[GitHub] [spark] pabloalcain commented on pull request #35045: [SPARK-37765][PYSPARK][WIP] DynamicDataFrame implementation

[GitBox <gi...@apache.org>]

pabloalcain commented on pull request #35045:
URL: https://github.com/apache/spark/pull/35045#issuecomment-1002546042


   I am 100% +1 with (and to be honest, would even prefer!) having the change on `pyspark.sql.DataFrame` API directly. To be honest, I feared that it might break some application code and introduce some skewness on the Scala and Python API, and that's why I thought it would be better to introduce this change first in an additive fashion so we disturb the least we can the current codebase. And, if this proves fruitful, we can port it to `DataFrame`. From the user's perspective, there would be no difference between the usage of `DynamicDataFrame` and the eventually-patched `DataFrame`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org