You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/09/25 12:20:20 UTC

[GitHub] [arrow-datafusion] Igosuki commented on issue #1048: Are different database systems going to be supported data sources?

Igosuki commented on issue #1048:
URL: https://github.com/apache/arrow-datafusion/issues/1048#issuecomment-927113320


   Hi, disclaimer not a maintainer. No it's not currently supported. I definitely see a use for it to join database tables with columnar data, although I find it is subject to the multiple-writer database anti-pattern, so you must have strong test suites to make sure changing the db (which is often owned by an API) won't affect the ETL process.
   One would have to create an experimental crate, implement a generic TableProvider for RDBMs systems, validate the SQL statements against the db, and adapt arrow to whatever data structure is used for sql statements.
   Interesting projects that come to mind for this are tokio-rs/rdbc and sqlx, there is also the arrow jdbc adapter and ongoing tasks like https://issues.apache.org/jira/browse/ARROW-7744


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org