[GitHub] [arrow] jorisvandenbossche commented on pull request #8912: ARROW-8221: [Python][Dataset] Expose schema inference/validation factory options through the validate_schema keyword

[GitBox <gi...@apache.org>]

jorisvandenbossche commented on pull request #8912:
URL: https://github.com/apache/arrow/pull/8912#issuecomment-820499099


   Rebased now.
   
   @lidavidm I am still a bit in doubt about the exact API. 
   For example, R has a `unify_schemas` keyword in their `open_dataset` function (https://arrow.apache.org/docs/r/reference/open_dataset.html). I find that name a bit more descriptive for the use case of "infer schema by checking all fragments", since then in practice it unifies the schema of all fragments (while with the current PR, you get that by using `validate_schema=True`). 
   On the other hand, R doesn't (currently) give a way to validate a specified schema, but not sure how useful that is in practice (since you would get the same error when starting to scan as well, if the specified schema doesn't match).
    
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org