Re: [I] create s3 fialed [arrow-datafusion]

["devinjdangelo (via GitHub)" <gi...@apache.org>]

devinjdangelo commented on issue #8421:
URL: https://github.com/apache/arrow-datafusion/issues/8421#issuecomment-1840729524

   The reason for the strict checking is to help identify typos or logical errors easily in SQL queries rather than silently ignoring invalid options. Given that datafusion-cli is using the same data structure to store additional options, there are a few options:
   
   - Add the additional datafusion-cli valid options to datafusion proper so they can be silently ignored rather than error
   - Have datafusion-cli remove any valid datafusion-cli specific options in the create external table statement before executing the statement
   - Downgrade the configuration error in datafusion to a warning message so execution doesn't fail
   
   I favor one of the first two options, but the third may be the best if we expect this problem to crop up beyond datafusion-cli. The question would be if it is likely for many downstream projects to inject additional options passed to a ListingTable or CopyTo statement.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org