[GitHub] [spark] rdblue edited a comment on pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

[GitBox <gi...@apache.org>]

rdblue edited a comment on pull request #27331:
URL: https://github.com/apache/spark/pull/27331#issuecomment-649809419


   > nobody knows the answer about the stability
   
   Short version: I don't think it is relevant; on its merits, I expect the API to be stable; I don't know what other people will do.
   
   I haven't replied because I don't see how it is an important concern. An experimental API in Scala can be an experimental API in Python. There's no strategy about not porting APIs that might change. And, it's easier to maintain compatibility on the Python side. That's why I think it should be merged, regardless of the question of stability.
   
   But if stability matters to you, I'll quickly try to address it.
   * Stability in an API is hard to judge without real-world use. That's why we wait some amount of time, instead of just declaring stable right away. I think this is very likely to be stable because it is a translation of the underlying SQL primitives that are stable. But...
   * I am not in control of the changes. Maintaining an API as stable is a choice; the previous DataFrameWriter is a poor API, but has been maintained as stable. In contrast, the 2.4 DSv2 API was good enough, but rewritten. I can't say what other members of the community might try to change in this API.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org