You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Chang She <ch...@eto.ai> on 2023/02/01 03:00:33 UTC
example pyarrow flight sql server?
Hi there,
I was looking to play around with flight sql server but couldn't find an
example/reference server implementation to help me get started. If anyone
can point me in the right direction I'd really appreciate it.
Thanks!
Chang
Re: example pyarrow flight sql server?
Posted by James Duong via user <us...@arrow.apache.org>.
Most applications have a generic ODBC or JDBC option that should be used if
there isn't one that is more specific to your database.
It is however popular to support PostgreSQL's SQL dialect, so if your
Flight SQL server is built on it, it might work well.
When building the JDBC driver, DataGrip was used fairly often for adhoc
testing. It was not tested with ODBC. SQL Workbench wasn't tested with
either driver.
The applications that ODBC was validated with can be seen in Dremio's
documentation:
https://docs.dremio.com/software/client-applications/clients/
On Wed., Feb. 1, 2023, 07:41 David Li, <li...@apache.org> wrote:
> Ah I'm not actually familiar with that - I imagine that once you install
> and configure the ODBC driver, you'd be able to select it directly from
> whatever client program.
>
> On Wed, Feb 1, 2023, at 10:37, Chang She wrote:
>
> Thank you both!
> Was just seeing if there was a way to be lazy :)
>
> David - thanks for the note on the odbc driver. I’m actually curious about
> the other end. Do you know if one can use the driver in say DataGrip / SQL
> workbench etc? If so, do we pick say “Postgres” connection or some other
> flavor?
>
>
> Cheers,
>
> Chang
>
> On Wed, Feb 1, 2023 at 7:34 AM Philip Moore <ph...@voltrondata.com>
> wrote:
>
> Ah yes – sorry I missed the “pyarrow” part in the subject line, my
> apologies. The example links are C++ only…
>
>
>
> *From: *David Li <li...@apache.org>
> *Date: *Wednesday, February 1, 2023 at 10:29 AM
> *To: *dl <us...@arrow.apache.org>
> *Subject: *Re: example pyarrow flight sql server?
>
> Flight SQL is not supported in Python (client or server).
>
>
>
> For the server side, I think we can consider PRs. The API surface is
> large, so I worry it would be tedious to maintain.
>
>
>
> For the client side, there is an experimental driver available for ADBC
> [1].
>
>
>
> For the ODBC driver, IIRC James Duong has stated it was mostly tested
> against Dremio. (It's not part of the Arrow project due to licensing
> issues, because it has code derived from a GPL codebase - though I believe
> those parts could be rewritten eventually.) So this is a case of 'try it
> and see'.
>
>
>
> [1]:
> https://github.com/apache/arrow-adbc/tree/main/python/adbc_driver_flightsql
>
>
>
> On Wed, Feb 1, 2023, at 09:00, Philip Moore wrote:
>
> Hi Chang,
>
>
>
> Here are the examples from the Arrow repo:
> https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example
>
>
>
> They have SQLite and Acero example servers built…
>
>
>
> Thanks.
>
>
>
> Phil
>
>
>
> *From: *Chang She <ch...@eto.ai>
> *Date: *Tuesday, January 31, 2023 at 10:01 PM
> *To: *user@arrow.apache.org <us...@arrow.apache.org>
> *Subject: *example pyarrow flight sql server?
>
> Hi there,
>
>
>
> I was looking to play around with flight sql server but couldn't find an
> example/reference server implementation to help me get started. If anyone
> can point me in the right direction I'd really appreciate it.
>
>
>
> Thanks!
>
>
>
> Chang
>
>
>
>
>
Re: example pyarrow flight sql server?
Posted by David Li <li...@apache.org>.
Ah I'm not actually familiar with that - I imagine that once you install and configure the ODBC driver, you'd be able to select it directly from whatever client program.
On Wed, Feb 1, 2023, at 10:37, Chang She wrote:
> Thank you both!
> Was just seeing if there was a way to be lazy :)
>
> David - thanks for the note on the odbc driver. I’m actually curious about the other end. Do you know if one can use the driver in say DataGrip / SQL workbench etc? If so, do we pick say “Postgres” connection or some other flavor?
>
>
> Cheers,
>
> Chang
>
> On Wed, Feb 1, 2023 at 7:34 AM Philip Moore <ph...@voltrondata.com> wrote:
>> Ah yes – sorry I missed the “pyarrow” part in the subject line, my apologies. The example links are C++ only…____
>> __ __
>> *From: *David Li <li...@apache.org>
>> *Date: *Wednesday, February 1, 2023 at 10:29 AM
>> *To: *dl <us...@arrow.apache.org>
>> *Subject: *Re: example pyarrow flight sql server?____
>> Flight SQL is not supported in Python (client or server).____
>> __ __
>> For the server side, I think we can consider PRs. The API surface is large, so I worry it would be tedious to maintain.____
>> __ __
>> For the client side, there is an experimental driver available for ADBC [1].____
>> __ __
>> For the ODBC driver, IIRC James Duong has stated it was mostly tested against Dremio. (It's not part of the Arrow project due to licensing issues, because it has code derived from a GPL codebase - though I believe those parts could be rewritten eventually.) So this is a case of 'try it and see'.____
>> __ __
>> [1]: https://github.com/apache/arrow-adbc/tree/main/python/adbc_driver_flightsql____
>> __ __
>> On Wed, Feb 1, 2023, at 09:00, Philip Moore wrote:____
>>> Hi Chang,____
>>>
>>> ____
>>>
>>> Here are the examples from the Arrow repo: https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example____
>>>
>>> ____
>>>
>>> They have SQLite and Acero example servers built…____
>>>
>>> ____
>>>
>>> Thanks.____
>>>
>>> ____
>>>
>>> Phil____
>>>
>>> ____
>>>
>>> *From: *Chang She <ch...@eto.ai>
>>> *Date: *Tuesday, January 31, 2023 at 10:01 PM
>>> *To: *user@arrow.apache.org <us...@arrow.apache.org>
>>> *Subject: *example pyarrow flight sql server?____
>>>
>>> Hi there,____
>>>
>>> ____
>>>
>>> I was looking to play around with flight sql server but couldn't find an example/reference server implementation to help me get started. If anyone can point me in the right direction I'd really appreciate it.____
>>>
>>> ____
>>>
>>> Thanks!____
>>>
>>> ____
>>>
>>> Chang____
>>>
>> __ __
Re: example pyarrow flight sql server?
Posted by Chang She <ch...@eto.ai>.
Thank you both!
Was just seeing if there was a way to be lazy :)
David - thanks for the note on the odbc driver. I’m actually curious about
the other end. Do you know if one can use the driver in say DataGrip / SQL
workbench etc? If so, do we pick say “Postgres” connection or some other
flavor?
Cheers,
Chang
On Wed, Feb 1, 2023 at 7:34 AM Philip Moore <ph...@voltrondata.com> wrote:
> Ah yes – sorry I missed the “pyarrow” part in the subject line, my
> apologies. The example links are C++ only…
>
>
>
> *From: *David Li <li...@apache.org>
> *Date: *Wednesday, February 1, 2023 at 10:29 AM
> *To: *dl <us...@arrow.apache.org>
> *Subject: *Re: example pyarrow flight sql server?
>
> Flight SQL is not supported in Python (client or server).
>
>
>
> For the server side, I think we can consider PRs. The API surface is
> large, so I worry it would be tedious to maintain.
>
>
>
> For the client side, there is an experimental driver available for ADBC
> [1].
>
>
>
> For the ODBC driver, IIRC James Duong has stated it was mostly tested
> against Dremio. (It's not part of the Arrow project due to licensing
> issues, because it has code derived from a GPL codebase - though I believe
> those parts could be rewritten eventually.) So this is a case of 'try it
> and see'.
>
>
>
> [1]:
> https://github.com/apache/arrow-adbc/tree/main/python/adbc_driver_flightsql
>
>
>
> On Wed, Feb 1, 2023, at 09:00, Philip Moore wrote:
>
> Hi Chang,
>
>
>
> Here are the examples from the Arrow repo:
> https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example
>
>
>
> They have SQLite and Acero example servers built…
>
>
>
> Thanks.
>
>
>
> Phil
>
>
>
> *From: *Chang She <ch...@eto.ai>
> *Date: *Tuesday, January 31, 2023 at 10:01 PM
> *To: *user@arrow.apache.org <us...@arrow.apache.org>
> *Subject: *example pyarrow flight sql server?
>
> Hi there,
>
>
>
> I was looking to play around with flight sql server but couldn't find an
> example/reference server implementation to help me get started. If anyone
> can point me in the right direction I'd really appreciate it.
>
>
>
> Thanks!
>
>
>
> Chang
>
>
>
Re: example pyarrow flight sql server?
Posted by Philip Moore <ph...@voltrondata.com>.
Ah yes – sorry I missed the “pyarrow” part in the subject line, my apologies. The example links are C++ only…
From: David Li <li...@apache.org>
Date: Wednesday, February 1, 2023 at 10:29 AM
To: dl <us...@arrow.apache.org>
Subject: Re: example pyarrow flight sql server?
Flight SQL is not supported in Python (client or server).
For the server side, I think we can consider PRs. The API surface is large, so I worry it would be tedious to maintain.
For the client side, there is an experimental driver available for ADBC [1].
For the ODBC driver, IIRC James Duong has stated it was mostly tested against Dremio. (It's not part of the Arrow project due to licensing issues, because it has code derived from a GPL codebase - though I believe those parts could be rewritten eventually.) So this is a case of 'try it and see'.
[1]: https://github.com/apache/arrow-adbc/tree/main/python/adbc_driver_flightsql
On Wed, Feb 1, 2023, at 09:00, Philip Moore wrote:
Hi Chang,
Here are the examples from the Arrow repo: https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example
They have SQLite and Acero example servers built…
Thanks.
Phil
From: Chang She <ch...@eto.ai>
Date: Tuesday, January 31, 2023 at 10:01 PM
To: user@arrow.apache.org <us...@arrow.apache.org>
Subject: example pyarrow flight sql server?
Hi there,
I was looking to play around with flight sql server but couldn't find an example/reference server implementation to help me get started. If anyone can point me in the right direction I'd really appreciate it.
Thanks!
Chang
Re: example pyarrow flight sql server?
Posted by David Li <li...@apache.org>.
Flight SQL is not supported in Python (client or server).
For the server side, I think we can consider PRs. The API surface is large, so I worry it would be tedious to maintain.
For the client side, there is an experimental driver available for ADBC [1].
For the ODBC driver, IIRC James Duong has stated it was mostly tested against Dremio. (It's not part of the Arrow project due to licensing issues, because it has code derived from a GPL codebase - though I believe those parts could be rewritten eventually.) So this is a case of 'try it and see'.
[1]: https://github.com/apache/arrow-adbc/tree/main/python/adbc_driver_flightsql
On Wed, Feb 1, 2023, at 09:00, Philip Moore wrote:
> Hi Chang,
>
> Here are the examples from the Arrow repo: https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example
>
> They have SQLite and Acero example servers built…
>
> Thanks.
>
> Phil
>
> *From: *Chang She <ch...@eto.ai>
> *Date: *Tuesday, January 31, 2023 at 10:01 PM
> *To: *user@arrow.apache.org <us...@arrow.apache.org>
> *Subject: *example pyarrow flight sql server?
> Hi there,
>
> I was looking to play around with flight sql server but couldn't find an example/reference server implementation to help me get started. If anyone can point me in the right direction I'd really appreciate it.
>
> Thanks!
>
> Chang
Re: example pyarrow flight sql server?
Posted by Philip Moore <ph...@voltrondata.com>.
Hi Chang,
Here are the examples from the Arrow repo: https://github.com/apache/arrow/tree/master/cpp/src/arrow/flight/sql/example
They have SQLite and Acero example servers built…
Thanks.
Phil
From: Chang She <ch...@eto.ai>
Date: Tuesday, January 31, 2023 at 10:01 PM
To: user@arrow.apache.org <us...@arrow.apache.org>
Subject: example pyarrow flight sql server?
Hi there,
I was looking to play around with flight sql server but couldn't find an example/reference server implementation to help me get started. If anyone can point me in the right direction I'd really appreciate it.
Thanks!
Chang
Re: example pyarrow flight sql server?
Posted by Chang She <ch...@eto.ai>.
Also, anyone know if Dremio’s ODBC driver for flight sql is compatible with
other sql ide’s? Thanks!
On Tue, Jan 31, 2023 at 7:00 PM Chang She <ch...@eto.ai> wrote:
> Hi there,
>
> I was looking to play around with flight sql server but couldn't find an
> example/reference server implementation to help me get started. If anyone
> can point me in the right direction I'd really appreciate it.
>
> Thanks!
>
> Chang
>