[GitHub] rdblue commented on issue #23829: [SPARK-26915][SQL]File source should write without schema validation in DataFrameWriter.save()

[GitBox <gi...@apache.org>]

rdblue commented on issue #23829: [SPARK-26915][SQL]File source should write without schema validation in DataFrameWriter.save()
URL: https://github.com/apache/spark/pull/23829#issuecomment-464808438
 
 
   @cloud-fan, one of the goals of v2 is to avoid special cases for internal sources. I think we should continue to avoid them.
   
   I'm happy to discuss a proposal for this, but I think we need a real proposal that has been thought through. Quick PRs to just-make-it-work-right-away are the reason we have unknown and unpredictable behavior in v1, and we need to be deliberate for what is introduced in v2.
   
   I think it is fine to have cases where validation is turned off, but those need to be well-defined. This is another reason to introduce a v2 write API where the behavior is obvious to the caller.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org