[GitHub] rdblue commented on issue #23714: [SPARK-26744][SQL]Support schema validation in FileDataSourceV2 framework

[GitBox <gi...@apache.org>]

rdblue commented on issue #23714: [SPARK-26744][SQL]Support schema validation in FileDataSourceV2 framework
URL: https://github.com/apache/spark/pull/23714#issuecomment-461875332
 
 
   @gengliangwang, why are you proposing to add this API that applies only to internal sources? Why not design this to work with all sources?
   
   I think you also need to be more clear about what you're trying to commit. What does this do? It sounds like it probably validates that a file format can stores a type. For example, can ORC support DECIMAL(44, 6)? That is generally useful. Why should it be a side API for internal sources?
   
   In short:
   * Please be clear in your description about what this commit does. What exactly does the validation do?
   * Please give a reason why it should apply only to internal sources given that a goal of the DSv2 API is to avoid special cases for internal sources.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org