[jira] [Commented] (ARROW-1780) JDBC Adapter for Apache Arrow

["Atul Dambalkar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369458#comment-16369458 ] 

Atul Dambalkar commented on ARROW-1780:
---------------------------------------

I have put together some very basic interface for the JDBC Adapter - so far by forking Arrow (https://github.com/atuldambalkar/arrow/tree/master/java/adapter/jdbc). I had a brief discussion with Uwe earlier on this on Slack, so wanted to get some more views on this and also not to redo or overstep. At this time, I have one API in the adapter which can return Arrow Vector objects after executing SQL query on the given JDBC connection object - VectorSchemaRoot sqlToArrow(Connection connection, String query).

One more possible interface could be to fetch a certain number of records from all the tables from the SQL database and build Arrow objects for that. The API can of-course be implemented lazily and only when the data for a particular table is requested.

> JDBC Adapter for Apache Arrow
> -----------------------------
>
>                 Key: ARROW-1780
>                 URL: https://issues.apache.org/jira/browse/ARROW-1780
>             Project: Apache Arrow
>          Issue Type: New Feature
>            Reporter: Atul Dambalkar
>            Priority: Major
>
> At a high level the JDBC Adapter will allow upstream apps to query RDBMS data over JDBC and get the JDBC objects converted to Arrow objects/structures. The upstream utility can then work with Arrow objects/structures with usual performance benefits. The utility will be very much similar to C++ implementation of "Convert a vector of row-wise data into an Arrow table" as described here - https://arrow.apache.org/docs/cpp/md_tutorials_row_wise_conversion.html
> The utility will read data from RDBMS and covert the data into Arrow objects/structures. So from that perspective this will Read data from RDBMS, If the utility can push Arrow objects to RDBMS is something need to be discussed and will be out of scope for this utility for now. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)