You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "berkaysynnada (via GitHub)" <gi...@apache.org> on 2025/05/05 13:02:32 UTC

[I] [DISCUSSION] Should `CREATE EXTERNAL TABLE's create a new file if it doesn't exist? [datafusion]

berkaysynnada opened a new issue, #15944:
URL: https://github.com/apache/datafusion/issues/15944

   `CREATE EXTERNAL TABLE t STORED AS CSV LOCATION '~/files/foo.csv';` gives an error if the given file does not exist. When we are trying to create a new file and then insert some values into it, we are forced to create the file externally. 
   
   I wonder how it sounds if `create external table`'s create the file indeed if it doesn't exist. If it seems like a useful behavior, then there will be a few more questions like
   - at what instance exactly the file is generated (during table provider generation, or during scan, or during the next select or insert into query?)
   - how are the stores other than local file system treated?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [DISCUSSION] Should `CREATE EXTERNAL TABLE's create a new file if it doesn't exist? [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #15944:
URL: https://github.com/apache/datafusion/issues/15944#issuecomment-2852313796

   I think there is a usecase for DataFusion remaining "read only" and so some users will prefer to get errors if the table doesn't exist
   
   Perhaps we can create a mode / config option, syntax to support "create file of not exists"
   
   ```sql
   CREATE EXTERNAL TABLE t STORED 
   AS CSV LOCATION '~/files/foo.csv'
   CREATE FILE IF NOT EXISTS -- <-- Proposed Syntax
   ```
   
   I don't like that particular syntax (hopefully someone could come up with something better) but I think it illustrates the idea


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org