You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/21 00:00:41 UTC

[GitHub] [arrow-datafusion] houqp commented on issue #1864: Add a DataFusion example capable of retrieving data from a custom datasource

houqp commented on issue #1864:
URL: https://github.com/apache/arrow-datafusion/issues/1864#issuecomment-1046351163


   @PierreZ https://github.com/delta-io/delta-rs/blob/2fa81c611e74235c13e5d27813d2e9d815d10ad9/rust/src/delta_datafusion.rs#L218 is an example of how to implement a custom datasource for datafusion through the `TableProvider` trait. This is the most efficient and recommended way.
   
   There is also a less involved hack, which is to read the data from your custom source into in memory Arrow record batches, then wrap them with a `MemTable`, here is an example of how I implemented a datasource for google spreadsheet using this approach: https://github.com/roapi/roapi/blob/cd271c65ac0769e41974f4c1cfe8499ab8e59bed/columnq/src/table/google_spreadsheets.rs#L300.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org