You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by David Li <li...@apache.org> on 2022/03/16 19:50:50 UTC
[VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Hello,
Jose Almeida and James Duong have proposed two additions to Arrow Flight SQL, an experimental protocol for interacting with SQL databases over Arrow Flight. The purpose of these additions is to provide necessary metadata for implementing a JDBC driver on top of Flight SQL [1].
The additions are as follows:
- As part of returned schemas, include metadata about the underlying SQL data type [2].
- Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the supported SQL data types [3].
Both pull requests implement the additions in C++ and Java and contain integration tests.
Please vote whether to accept these enhancements. The vote will be open for at least 72 hours.
[ ] +1 Accept these protocol additions
[ ] +0
[ ] -1 Do not accept these protocol additions because…
[1]: https://github.com/apache/arrow/pull/12254
[2]: https://github.com/apache/arrow/pull/11999
[3]: https://github.com/apache/arrow/pull/11982
-David
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Bryan Cutler <cu...@gmail.com>.
+1 (non-binding)
On Mon, Mar 28, 2022, 7:07 AM Andrew Lamb <al...@influxdata.com> wrote:
> Thank you David for pushing this through -- I think the overall FlightSQL
> story is very compelling for the Arrow ecosystem
>
> I am also +1 on the idea, but I haven't had enough time to study the
> implementation in detail yet. I hope to do so over the next few weeks
>
> Andrew
>
> On Sun, Mar 27, 2022 at 7:06 PM David Li <li...@apache.org> wrote:
>
> > Thanks Wes, and sorry about the lapse.
> >
> > On Sun, Mar 27, 2022, at 14:04, Wes McKinney wrote:
> > > Adding my +1 (binding) vote (technically votes need 3 binding +1's so
> > > this will pass)
> > >
> > > On Fri, Mar 25, 2022 at 4:12 PM David Li <li...@apache.org> wrote:
> > >>
> > >> The vote has been open for a while now without objection, so the vote
> > passes with 2 +1 votes (binding), 4 +1 votes (non-binding).
> > >>
> > >> Thanks to all the contributors and reviewers who worked on these
> > changes.
> > >>
> > >> On Wed, Mar 23, 2022, at 13:28, José Almeida wrote:
> > >> > Thanks for the reply David. Your answer is correct.
> > >> >
> > >> > The first PR [1], we are not voting for it yet. It contains what
> we've
> > >> > built from the JDBC using flight-sql so far. I don't recall if we
> > already
> > >> > implemented the proposals from PR [2] and [3].
> > >> > I guess that we already have a draft of typeInfo and ColumnMetadata
> on
> > >> > JDBC, but they will need changes after this is approved.
> > >> > Feel free to take a look in the JDBC PR and give us your feedback
> > Andrew.
> > >> > All feedbacks are welcome 😀
> > >> >
> > >> > The second PR [2] contains the metadata related to the columns, so
> > some
> > >> > operations will be able to send it as response and the JDBC/ODBC
> will
> > have
> > >> > access to it. The metadata that we are sending
> > >> > were the ones we identified, but perhaps there should be more that
> we
> > >> > couldn't identify.
> > >> >
> > >> > The third PR[3] contains another functionality that retrieves
> > information
> > >> > about the types that the data sources support.
> > >> >
> > >> > Feel free to ask any questions you might have 😀
> > >> >
> > >> > On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org>
> > wrote:
> > >> >
> > >> >> Maybe one of the contributors wants to chime in with more details,
> > but:
> > >> >>
> > >> >> PR#12254 isn't part of the vote, it's just the motivation for these
> > >> >> changes. I suppose it isn't fully in sync with the other PRs?
> > >> >> PR#11999 annotates fields with metadata that is used to support
> > JDBC/ODBC
> > >> >> drivers (e.g. the ability to tell what table a column originated
> > from)
> > >> >> PR#11982 is used to retrieve metadata about supported SQL data
> types.
> > >> >>
> > >> >> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
> > >> >> > BTW thank you all for your work in this matter (making JDBC/ODBC
> > >> >> clients)!
> > >> >> > I think it is super valuable for the overall ecosystem.
> > >> >> >
> > >> >> > I am sorry for missing the conversation, but I am not clear on
> > what we
> > >> >> are
> > >> >> > voting on. Can we please clarify what changes are proposed to
> > FlightSQL?
> > >> >> >
> > >> >> > The PRs appear to contain changes to FlightSql.proto that seem
> > somewhat
> > >> >> > redundant / contradictory. For example:
> > >> >> >
> > >> >> > Metadata named `CATALOG_NAME` on [1]
> > >> >> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
> > >> >> > No metadata for catalog name on [3] (but does have other metadata
> > like
> > >> >> > auto_increment)
> > >> >> >
> > >> >> > Andrew
> > >> >> >
> > >> >> > [1] https://github.com/apache/arrow/pull/12254
> > >> >> > [2] https://github.com/apache/arrow/pull/11999/
> > >> >> > [3] https://github.com/apache/arrow/pull/11982
> > >> >> >
> > >> >> >
> > >> >> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <
> antoine@python.org
> > >
> > >> >> wrote:
> > >> >> >
> > >> >> >>
> > >> >> >> Moral +1 from me. I've posted minor comments on the specs
> changes
> > in the
> > >> >> >> PRs.
> > >> >> >>
> > >> >> >>
> > >> >> >> Le 16/03/2022 à 20:50, David Li a écrit :
> > >> >> >> > Hello,
> > >> >> >> >
> > >> >> >> > Jose Almeida and James Duong have proposed two additions to
> > Arrow
> > >> >> Flight
> > >> >> >> SQL, an experimental protocol for interacting with SQL databases
> > over
> > >> >> Arrow
> > >> >> >> Flight. The purpose of these additions is to provide necessary
> > metadata
> > >> >> for
> > >> >> >> implementing a JDBC driver on top of Flight SQL [1].
> > >> >> >> >
> > >> >> >> > The additions are as follows:
> > >> >> >> >
> > >> >> >> > - As part of returned schemas, include metadata about the
> > underlying
> > >> >> SQL
> > >> >> >> data type [2].
> > >> >> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata
> > about the
> > >> >> >> supported SQL data types [3].
> > >> >> >> >
> > >> >> >> > Both pull requests implement the additions in C++ and Java and
> > contain
> > >> >> >> integration tests.
> > >> >> >> >
> > >> >> >> > Please vote whether to accept these enhancements. The vote
> will
> > be
> > >> >> open
> > >> >> >> for at least 72 hours.
> > >> >> >> >
> > >> >> >> > [ ] +1 Accept these protocol additions
> > >> >> >> > [ ] +0
> > >> >> >> > [ ] -1 Do not accept these protocol additions because…
> > >> >> >> >
> > >> >> >> > [1]: https://github.com/apache/arrow/pull/12254
> > >> >> >> > [2]: https://github.com/apache/arrow/pull/11999
> > >> >> >> > [3]: https://github.com/apache/arrow/pull/11982
> > >> >> >> >
> > >> >> >> > -David
> > >> >> >>
> > >> >>
> >
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Andrew Lamb <al...@influxdata.com>.
Thank you David for pushing this through -- I think the overall FlightSQL
story is very compelling for the Arrow ecosystem
I am also +1 on the idea, but I haven't had enough time to study the
implementation in detail yet. I hope to do so over the next few weeks
Andrew
On Sun, Mar 27, 2022 at 7:06 PM David Li <li...@apache.org> wrote:
> Thanks Wes, and sorry about the lapse.
>
> On Sun, Mar 27, 2022, at 14:04, Wes McKinney wrote:
> > Adding my +1 (binding) vote (technically votes need 3 binding +1's so
> > this will pass)
> >
> > On Fri, Mar 25, 2022 at 4:12 PM David Li <li...@apache.org> wrote:
> >>
> >> The vote has been open for a while now without objection, so the vote
> passes with 2 +1 votes (binding), 4 +1 votes (non-binding).
> >>
> >> Thanks to all the contributors and reviewers who worked on these
> changes.
> >>
> >> On Wed, Mar 23, 2022, at 13:28, José Almeida wrote:
> >> > Thanks for the reply David. Your answer is correct.
> >> >
> >> > The first PR [1], we are not voting for it yet. It contains what we've
> >> > built from the JDBC using flight-sql so far. I don't recall if we
> already
> >> > implemented the proposals from PR [2] and [3].
> >> > I guess that we already have a draft of typeInfo and ColumnMetadata on
> >> > JDBC, but they will need changes after this is approved.
> >> > Feel free to take a look in the JDBC PR and give us your feedback
> Andrew.
> >> > All feedbacks are welcome 😀
> >> >
> >> > The second PR [2] contains the metadata related to the columns, so
> some
> >> > operations will be able to send it as response and the JDBC/ODBC will
> have
> >> > access to it. The metadata that we are sending
> >> > were the ones we identified, but perhaps there should be more that we
> >> > couldn't identify.
> >> >
> >> > The third PR[3] contains another functionality that retrieves
> information
> >> > about the types that the data sources support.
> >> >
> >> > Feel free to ask any questions you might have 😀
> >> >
> >> > On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org>
> wrote:
> >> >
> >> >> Maybe one of the contributors wants to chime in with more details,
> but:
> >> >>
> >> >> PR#12254 isn't part of the vote, it's just the motivation for these
> >> >> changes. I suppose it isn't fully in sync with the other PRs?
> >> >> PR#11999 annotates fields with metadata that is used to support
> JDBC/ODBC
> >> >> drivers (e.g. the ability to tell what table a column originated
> from)
> >> >> PR#11982 is used to retrieve metadata about supported SQL data types.
> >> >>
> >> >> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
> >> >> > BTW thank you all for your work in this matter (making JDBC/ODBC
> >> >> clients)!
> >> >> > I think it is super valuable for the overall ecosystem.
> >> >> >
> >> >> > I am sorry for missing the conversation, but I am not clear on
> what we
> >> >> are
> >> >> > voting on. Can we please clarify what changes are proposed to
> FlightSQL?
> >> >> >
> >> >> > The PRs appear to contain changes to FlightSql.proto that seem
> somewhat
> >> >> > redundant / contradictory. For example:
> >> >> >
> >> >> > Metadata named `CATALOG_NAME` on [1]
> >> >> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
> >> >> > No metadata for catalog name on [3] (but does have other metadata
> like
> >> >> > auto_increment)
> >> >> >
> >> >> > Andrew
> >> >> >
> >> >> > [1] https://github.com/apache/arrow/pull/12254
> >> >> > [2] https://github.com/apache/arrow/pull/11999/
> >> >> > [3] https://github.com/apache/arrow/pull/11982
> >> >> >
> >> >> >
> >> >> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <antoine@python.org
> >
> >> >> wrote:
> >> >> >
> >> >> >>
> >> >> >> Moral +1 from me. I've posted minor comments on the specs changes
> in the
> >> >> >> PRs.
> >> >> >>
> >> >> >>
> >> >> >> Le 16/03/2022 à 20:50, David Li a écrit :
> >> >> >> > Hello,
> >> >> >> >
> >> >> >> > Jose Almeida and James Duong have proposed two additions to
> Arrow
> >> >> Flight
> >> >> >> SQL, an experimental protocol for interacting with SQL databases
> over
> >> >> Arrow
> >> >> >> Flight. The purpose of these additions is to provide necessary
> metadata
> >> >> for
> >> >> >> implementing a JDBC driver on top of Flight SQL [1].
> >> >> >> >
> >> >> >> > The additions are as follows:
> >> >> >> >
> >> >> >> > - As part of returned schemas, include metadata about the
> underlying
> >> >> SQL
> >> >> >> data type [2].
> >> >> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata
> about the
> >> >> >> supported SQL data types [3].
> >> >> >> >
> >> >> >> > Both pull requests implement the additions in C++ and Java and
> contain
> >> >> >> integration tests.
> >> >> >> >
> >> >> >> > Please vote whether to accept these enhancements. The vote will
> be
> >> >> open
> >> >> >> for at least 72 hours.
> >> >> >> >
> >> >> >> > [ ] +1 Accept these protocol additions
> >> >> >> > [ ] +0
> >> >> >> > [ ] -1 Do not accept these protocol additions because…
> >> >> >> >
> >> >> >> > [1]: https://github.com/apache/arrow/pull/12254
> >> >> >> > [2]: https://github.com/apache/arrow/pull/11999
> >> >> >> > [3]: https://github.com/apache/arrow/pull/11982
> >> >> >> >
> >> >> >> > -David
> >> >> >>
> >> >>
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by David Li <li...@apache.org>.
Thanks Wes, and sorry about the lapse.
On Sun, Mar 27, 2022, at 14:04, Wes McKinney wrote:
> Adding my +1 (binding) vote (technically votes need 3 binding +1's so
> this will pass)
>
> On Fri, Mar 25, 2022 at 4:12 PM David Li <li...@apache.org> wrote:
>>
>> The vote has been open for a while now without objection, so the vote passes with 2 +1 votes (binding), 4 +1 votes (non-binding).
>>
>> Thanks to all the contributors and reviewers who worked on these changes.
>>
>> On Wed, Mar 23, 2022, at 13:28, José Almeida wrote:
>> > Thanks for the reply David. Your answer is correct.
>> >
>> > The first PR [1], we are not voting for it yet. It contains what we've
>> > built from the JDBC using flight-sql so far. I don't recall if we already
>> > implemented the proposals from PR [2] and [3].
>> > I guess that we already have a draft of typeInfo and ColumnMetadata on
>> > JDBC, but they will need changes after this is approved.
>> > Feel free to take a look in the JDBC PR and give us your feedback Andrew.
>> > All feedbacks are welcome 😀
>> >
>> > The second PR [2] contains the metadata related to the columns, so some
>> > operations will be able to send it as response and the JDBC/ODBC will have
>> > access to it. The metadata that we are sending
>> > were the ones we identified, but perhaps there should be more that we
>> > couldn't identify.
>> >
>> > The third PR[3] contains another functionality that retrieves information
>> > about the types that the data sources support.
>> >
>> > Feel free to ask any questions you might have 😀
>> >
>> > On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org> wrote:
>> >
>> >> Maybe one of the contributors wants to chime in with more details, but:
>> >>
>> >> PR#12254 isn't part of the vote, it's just the motivation for these
>> >> changes. I suppose it isn't fully in sync with the other PRs?
>> >> PR#11999 annotates fields with metadata that is used to support JDBC/ODBC
>> >> drivers (e.g. the ability to tell what table a column originated from)
>> >> PR#11982 is used to retrieve metadata about supported SQL data types.
>> >>
>> >> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
>> >> > BTW thank you all for your work in this matter (making JDBC/ODBC
>> >> clients)!
>> >> > I think it is super valuable for the overall ecosystem.
>> >> >
>> >> > I am sorry for missing the conversation, but I am not clear on what we
>> >> are
>> >> > voting on. Can we please clarify what changes are proposed to FlightSQL?
>> >> >
>> >> > The PRs appear to contain changes to FlightSql.proto that seem somewhat
>> >> > redundant / contradictory. For example:
>> >> >
>> >> > Metadata named `CATALOG_NAME` on [1]
>> >> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
>> >> > No metadata for catalog name on [3] (but does have other metadata like
>> >> > auto_increment)
>> >> >
>> >> > Andrew
>> >> >
>> >> > [1] https://github.com/apache/arrow/pull/12254
>> >> > [2] https://github.com/apache/arrow/pull/11999/
>> >> > [3] https://github.com/apache/arrow/pull/11982
>> >> >
>> >> >
>> >> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org>
>> >> wrote:
>> >> >
>> >> >>
>> >> >> Moral +1 from me. I've posted minor comments on the specs changes in the
>> >> >> PRs.
>> >> >>
>> >> >>
>> >> >> Le 16/03/2022 à 20:50, David Li a écrit :
>> >> >> > Hello,
>> >> >> >
>> >> >> > Jose Almeida and James Duong have proposed two additions to Arrow
>> >> Flight
>> >> >> SQL, an experimental protocol for interacting with SQL databases over
>> >> Arrow
>> >> >> Flight. The purpose of these additions is to provide necessary metadata
>> >> for
>> >> >> implementing a JDBC driver on top of Flight SQL [1].
>> >> >> >
>> >> >> > The additions are as follows:
>> >> >> >
>> >> >> > - As part of returned schemas, include metadata about the underlying
>> >> SQL
>> >> >> data type [2].
>> >> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
>> >> >> supported SQL data types [3].
>> >> >> >
>> >> >> > Both pull requests implement the additions in C++ and Java and contain
>> >> >> integration tests.
>> >> >> >
>> >> >> > Please vote whether to accept these enhancements. The vote will be
>> >> open
>> >> >> for at least 72 hours.
>> >> >> >
>> >> >> > [ ] +1 Accept these protocol additions
>> >> >> > [ ] +0
>> >> >> > [ ] -1 Do not accept these protocol additions because…
>> >> >> >
>> >> >> > [1]: https://github.com/apache/arrow/pull/12254
>> >> >> > [2]: https://github.com/apache/arrow/pull/11999
>> >> >> > [3]: https://github.com/apache/arrow/pull/11982
>> >> >> >
>> >> >> > -David
>> >> >>
>> >>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Wes McKinney <we...@gmail.com>.
Adding my +1 (binding) vote (technically votes need 3 binding +1's so
this will pass)
On Fri, Mar 25, 2022 at 4:12 PM David Li <li...@apache.org> wrote:
>
> The vote has been open for a while now without objection, so the vote passes with 2 +1 votes (binding), 4 +1 votes (non-binding).
>
> Thanks to all the contributors and reviewers who worked on these changes.
>
> On Wed, Mar 23, 2022, at 13:28, José Almeida wrote:
> > Thanks for the reply David. Your answer is correct.
> >
> > The first PR [1], we are not voting for it yet. It contains what we've
> > built from the JDBC using flight-sql so far. I don't recall if we already
> > implemented the proposals from PR [2] and [3].
> > I guess that we already have a draft of typeInfo and ColumnMetadata on
> > JDBC, but they will need changes after this is approved.
> > Feel free to take a look in the JDBC PR and give us your feedback Andrew.
> > All feedbacks are welcome 😀
> >
> > The second PR [2] contains the metadata related to the columns, so some
> > operations will be able to send it as response and the JDBC/ODBC will have
> > access to it. The metadata that we are sending
> > were the ones we identified, but perhaps there should be more that we
> > couldn't identify.
> >
> > The third PR[3] contains another functionality that retrieves information
> > about the types that the data sources support.
> >
> > Feel free to ask any questions you might have 😀
> >
> > On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org> wrote:
> >
> >> Maybe one of the contributors wants to chime in with more details, but:
> >>
> >> PR#12254 isn't part of the vote, it's just the motivation for these
> >> changes. I suppose it isn't fully in sync with the other PRs?
> >> PR#11999 annotates fields with metadata that is used to support JDBC/ODBC
> >> drivers (e.g. the ability to tell what table a column originated from)
> >> PR#11982 is used to retrieve metadata about supported SQL data types.
> >>
> >> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
> >> > BTW thank you all for your work in this matter (making JDBC/ODBC
> >> clients)!
> >> > I think it is super valuable for the overall ecosystem.
> >> >
> >> > I am sorry for missing the conversation, but I am not clear on what we
> >> are
> >> > voting on. Can we please clarify what changes are proposed to FlightSQL?
> >> >
> >> > The PRs appear to contain changes to FlightSql.proto that seem somewhat
> >> > redundant / contradictory. For example:
> >> >
> >> > Metadata named `CATALOG_NAME` on [1]
> >> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
> >> > No metadata for catalog name on [3] (but does have other metadata like
> >> > auto_increment)
> >> >
> >> > Andrew
> >> >
> >> > [1] https://github.com/apache/arrow/pull/12254
> >> > [2] https://github.com/apache/arrow/pull/11999/
> >> > [3] https://github.com/apache/arrow/pull/11982
> >> >
> >> >
> >> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org>
> >> wrote:
> >> >
> >> >>
> >> >> Moral +1 from me. I've posted minor comments on the specs changes in the
> >> >> PRs.
> >> >>
> >> >>
> >> >> Le 16/03/2022 à 20:50, David Li a écrit :
> >> >> > Hello,
> >> >> >
> >> >> > Jose Almeida and James Duong have proposed two additions to Arrow
> >> Flight
> >> >> SQL, an experimental protocol for interacting with SQL databases over
> >> Arrow
> >> >> Flight. The purpose of these additions is to provide necessary metadata
> >> for
> >> >> implementing a JDBC driver on top of Flight SQL [1].
> >> >> >
> >> >> > The additions are as follows:
> >> >> >
> >> >> > - As part of returned schemas, include metadata about the underlying
> >> SQL
> >> >> data type [2].
> >> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> >> >> supported SQL data types [3].
> >> >> >
> >> >> > Both pull requests implement the additions in C++ and Java and contain
> >> >> integration tests.
> >> >> >
> >> >> > Please vote whether to accept these enhancements. The vote will be
> >> open
> >> >> for at least 72 hours.
> >> >> >
> >> >> > [ ] +1 Accept these protocol additions
> >> >> > [ ] +0
> >> >> > [ ] -1 Do not accept these protocol additions because…
> >> >> >
> >> >> > [1]: https://github.com/apache/arrow/pull/12254
> >> >> > [2]: https://github.com/apache/arrow/pull/11999
> >> >> > [3]: https://github.com/apache/arrow/pull/11982
> >> >> >
> >> >> > -David
> >> >>
> >>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by David Li <li...@apache.org>.
The vote has been open for a while now without objection, so the vote passes with 2 +1 votes (binding), 4 +1 votes (non-binding).
Thanks to all the contributors and reviewers who worked on these changes.
On Wed, Mar 23, 2022, at 13:28, José Almeida wrote:
> Thanks for the reply David. Your answer is correct.
>
> The first PR [1], we are not voting for it yet. It contains what we've
> built from the JDBC using flight-sql so far. I don't recall if we already
> implemented the proposals from PR [2] and [3].
> I guess that we already have a draft of typeInfo and ColumnMetadata on
> JDBC, but they will need changes after this is approved.
> Feel free to take a look in the JDBC PR and give us your feedback Andrew.
> All feedbacks are welcome 😀
>
> The second PR [2] contains the metadata related to the columns, so some
> operations will be able to send it as response and the JDBC/ODBC will have
> access to it. The metadata that we are sending
> were the ones we identified, but perhaps there should be more that we
> couldn't identify.
>
> The third PR[3] contains another functionality that retrieves information
> about the types that the data sources support.
>
> Feel free to ask any questions you might have 😀
>
> On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org> wrote:
>
>> Maybe one of the contributors wants to chime in with more details, but:
>>
>> PR#12254 isn't part of the vote, it's just the motivation for these
>> changes. I suppose it isn't fully in sync with the other PRs?
>> PR#11999 annotates fields with metadata that is used to support JDBC/ODBC
>> drivers (e.g. the ability to tell what table a column originated from)
>> PR#11982 is used to retrieve metadata about supported SQL data types.
>>
>> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
>> > BTW thank you all for your work in this matter (making JDBC/ODBC
>> clients)!
>> > I think it is super valuable for the overall ecosystem.
>> >
>> > I am sorry for missing the conversation, but I am not clear on what we
>> are
>> > voting on. Can we please clarify what changes are proposed to FlightSQL?
>> >
>> > The PRs appear to contain changes to FlightSql.proto that seem somewhat
>> > redundant / contradictory. For example:
>> >
>> > Metadata named `CATALOG_NAME` on [1]
>> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
>> > No metadata for catalog name on [3] (but does have other metadata like
>> > auto_increment)
>> >
>> > Andrew
>> >
>> > [1] https://github.com/apache/arrow/pull/12254
>> > [2] https://github.com/apache/arrow/pull/11999/
>> > [3] https://github.com/apache/arrow/pull/11982
>> >
>> >
>> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org>
>> wrote:
>> >
>> >>
>> >> Moral +1 from me. I've posted minor comments on the specs changes in the
>> >> PRs.
>> >>
>> >>
>> >> Le 16/03/2022 à 20:50, David Li a écrit :
>> >> > Hello,
>> >> >
>> >> > Jose Almeida and James Duong have proposed two additions to Arrow
>> Flight
>> >> SQL, an experimental protocol for interacting with SQL databases over
>> Arrow
>> >> Flight. The purpose of these additions is to provide necessary metadata
>> for
>> >> implementing a JDBC driver on top of Flight SQL [1].
>> >> >
>> >> > The additions are as follows:
>> >> >
>> >> > - As part of returned schemas, include metadata about the underlying
>> SQL
>> >> data type [2].
>> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
>> >> supported SQL data types [3].
>> >> >
>> >> > Both pull requests implement the additions in C++ and Java and contain
>> >> integration tests.
>> >> >
>> >> > Please vote whether to accept these enhancements. The vote will be
>> open
>> >> for at least 72 hours.
>> >> >
>> >> > [ ] +1 Accept these protocol additions
>> >> > [ ] +0
>> >> > [ ] -1 Do not accept these protocol additions because…
>> >> >
>> >> > [1]: https://github.com/apache/arrow/pull/12254
>> >> > [2]: https://github.com/apache/arrow/pull/11999
>> >> > [3]: https://github.com/apache/arrow/pull/11982
>> >> >
>> >> > -David
>> >>
>>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by José Almeida <jo...@simbioseventures.com>.
Thanks for the reply David. Your answer is correct.
The first PR [1], we are not voting for it yet. It contains what we've
built from the JDBC using flight-sql so far. I don't recall if we already
implemented the proposals from PR [2] and [3].
I guess that we already have a draft of typeInfo and ColumnMetadata on
JDBC, but they will need changes after this is approved.
Feel free to take a look in the JDBC PR and give us your feedback Andrew.
All feedbacks are welcome 😀
The second PR [2] contains the metadata related to the columns, so some
operations will be able to send it as response and the JDBC/ODBC will have
access to it. The metadata that we are sending
were the ones we identified, but perhaps there should be more that we
couldn't identify.
The third PR[3] contains another functionality that retrieves information
about the types that the data sources support.
Feel free to ask any questions you might have 😀
On Tue, Mar 22, 2022 at 10:13 AM David Li <li...@apache.org> wrote:
> Maybe one of the contributors wants to chime in with more details, but:
>
> PR#12254 isn't part of the vote, it's just the motivation for these
> changes. I suppose it isn't fully in sync with the other PRs?
> PR#11999 annotates fields with metadata that is used to support JDBC/ODBC
> drivers (e.g. the ability to tell what table a column originated from)
> PR#11982 is used to retrieve metadata about supported SQL data types.
>
> On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
> > BTW thank you all for your work in this matter (making JDBC/ODBC
> clients)!
> > I think it is super valuable for the overall ecosystem.
> >
> > I am sorry for missing the conversation, but I am not clear on what we
> are
> > voting on. Can we please clarify what changes are proposed to FlightSQL?
> >
> > The PRs appear to contain changes to FlightSql.proto that seem somewhat
> > redundant / contradictory. For example:
> >
> > Metadata named `CATALOG_NAME` on [1]
> > Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
> > No metadata for catalog name on [3] (but does have other metadata like
> > auto_increment)
> >
> > Andrew
> >
> > [1] https://github.com/apache/arrow/pull/12254
> > [2] https://github.com/apache/arrow/pull/11999/
> > [3] https://github.com/apache/arrow/pull/11982
> >
> >
> > On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org>
> wrote:
> >
> >>
> >> Moral +1 from me. I've posted minor comments on the specs changes in the
> >> PRs.
> >>
> >>
> >> Le 16/03/2022 à 20:50, David Li a écrit :
> >> > Hello,
> >> >
> >> > Jose Almeida and James Duong have proposed two additions to Arrow
> Flight
> >> SQL, an experimental protocol for interacting with SQL databases over
> Arrow
> >> Flight. The purpose of these additions is to provide necessary metadata
> for
> >> implementing a JDBC driver on top of Flight SQL [1].
> >> >
> >> > The additions are as follows:
> >> >
> >> > - As part of returned schemas, include metadata about the underlying
> SQL
> >> data type [2].
> >> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> >> supported SQL data types [3].
> >> >
> >> > Both pull requests implement the additions in C++ and Java and contain
> >> integration tests.
> >> >
> >> > Please vote whether to accept these enhancements. The vote will be
> open
> >> for at least 72 hours.
> >> >
> >> > [ ] +1 Accept these protocol additions
> >> > [ ] +0
> >> > [ ] -1 Do not accept these protocol additions because…
> >> >
> >> > [1]: https://github.com/apache/arrow/pull/12254
> >> > [2]: https://github.com/apache/arrow/pull/11999
> >> > [3]: https://github.com/apache/arrow/pull/11982
> >> >
> >> > -David
> >>
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by David Li <li...@apache.org>.
Maybe one of the contributors wants to chime in with more details, but:
PR#12254 isn't part of the vote, it's just the motivation for these changes. I suppose it isn't fully in sync with the other PRs?
PR#11999 annotates fields with metadata that is used to support JDBC/ODBC drivers (e.g. the ability to tell what table a column originated from)
PR#11982 is used to retrieve metadata about supported SQL data types.
On Mon, Mar 21, 2022, at 16:08, Andrew Lamb wrote:
> BTW thank you all for your work in this matter (making JDBC/ODBC clients)!
> I think it is super valuable for the overall ecosystem.
>
> I am sorry for missing the conversation, but I am not clear on what we are
> voting on. Can we please clarify what changes are proposed to FlightSQL?
>
> The PRs appear to contain changes to FlightSql.proto that seem somewhat
> redundant / contradictory. For example:
>
> Metadata named `CATALOG_NAME` on [1]
> Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
> No metadata for catalog name on [3] (but does have other metadata like
> auto_increment)
>
> Andrew
>
> [1] https://github.com/apache/arrow/pull/12254
> [2] https://github.com/apache/arrow/pull/11999/
> [3] https://github.com/apache/arrow/pull/11982
>
>
> On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org> wrote:
>
>>
>> Moral +1 from me. I've posted minor comments on the specs changes in the
>> PRs.
>>
>>
>> Le 16/03/2022 à 20:50, David Li a écrit :
>> > Hello,
>> >
>> > Jose Almeida and James Duong have proposed two additions to Arrow Flight
>> SQL, an experimental protocol for interacting with SQL databases over Arrow
>> Flight. The purpose of these additions is to provide necessary metadata for
>> implementing a JDBC driver on top of Flight SQL [1].
>> >
>> > The additions are as follows:
>> >
>> > - As part of returned schemas, include metadata about the underlying SQL
>> data type [2].
>> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
>> supported SQL data types [3].
>> >
>> > Both pull requests implement the additions in C++ and Java and contain
>> integration tests.
>> >
>> > Please vote whether to accept these enhancements. The vote will be open
>> for at least 72 hours.
>> >
>> > [ ] +1 Accept these protocol additions
>> > [ ] +0
>> > [ ] -1 Do not accept these protocol additions because…
>> >
>> > [1]: https://github.com/apache/arrow/pull/12254
>> > [2]: https://github.com/apache/arrow/pull/11999
>> > [3]: https://github.com/apache/arrow/pull/11982
>> >
>> > -David
>>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Andrew Lamb <al...@influxdata.com>.
BTW thank you all for your work in this matter (making JDBC/ODBC clients)!
I think it is super valuable for the overall ecosystem.
I am sorry for missing the conversation, but I am not clear on what we are
voting on. Can we please clarify what changes are proposed to FlightSQL?
The PRs appear to contain changes to FlightSql.proto that seem somewhat
redundant / contradictory. For example:
Metadata named `CATALOG_NAME` on [1]
Metadata named `ARROW:FLIGHT:SQL:CATALOG_NAME` on [2]
No metadata for catalog name on [3] (but does have other metadata like
auto_increment)
Andrew
[1] https://github.com/apache/arrow/pull/12254
[2] https://github.com/apache/arrow/pull/11999/
[3] https://github.com/apache/arrow/pull/11982
On Mon, Mar 21, 2022 at 2:02 PM Antoine Pitrou <an...@python.org> wrote:
>
> Moral +1 from me. I've posted minor comments on the specs changes in the
> PRs.
>
>
> Le 16/03/2022 à 20:50, David Li a écrit :
> > Hello,
> >
> > Jose Almeida and James Duong have proposed two additions to Arrow Flight
> SQL, an experimental protocol for interacting with SQL databases over Arrow
> Flight. The purpose of these additions is to provide necessary metadata for
> implementing a JDBC driver on top of Flight SQL [1].
> >
> > The additions are as follows:
> >
> > - As part of returned schemas, include metadata about the underlying SQL
> data type [2].
> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> supported SQL data types [3].
> >
> > Both pull requests implement the additions in C++ and Java and contain
> integration tests.
> >
> > Please vote whether to accept these enhancements. The vote will be open
> for at least 72 hours.
> >
> > [ ] +1 Accept these protocol additions
> > [ ] +0
> > [ ] -1 Do not accept these protocol additions because…
> >
> > [1]: https://github.com/apache/arrow/pull/12254
> > [2]: https://github.com/apache/arrow/pull/11999
> > [3]: https://github.com/apache/arrow/pull/11982
> >
> > -David
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Antoine Pitrou <an...@python.org>.
Moral +1 from me. I've posted minor comments on the specs changes in the
PRs.
Le 16/03/2022 à 20:50, David Li a écrit :
> Hello,
>
> Jose Almeida and James Duong have proposed two additions to Arrow Flight SQL, an experimental protocol for interacting with SQL databases over Arrow Flight. The purpose of these additions is to provide necessary metadata for implementing a JDBC driver on top of Flight SQL [1].
>
> The additions are as follows:
>
> - As part of returned schemas, include metadata about the underlying SQL data type [2].
> - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the supported SQL data types [3].
>
> Both pull requests implement the additions in C++ and Java and contain integration tests.
>
> Please vote whether to accept these enhancements. The vote will be open for at least 72 hours.
>
> [ ] +1 Accept these protocol additions
> [ ] +0
> [ ] -1 Do not accept these protocol additions because…
>
> [1]: https://github.com/apache/arrow/pull/12254
> [2]: https://github.com/apache/arrow/pull/11999
> [3]: https://github.com/apache/arrow/pull/11982
>
> -David
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by "Gabriel Escobar (CW)" <ga...@dremio.com>.
+1 (non-binding)
On Mon, Mar 21, 2022 at 2:54 PM Vinicius Fraga <sx...@gmail.com> wrote:
> +1 (non-binding)
>
> On Mon, 21 Mar 2022 at 14:53, Rafael Telles <ra...@telles.dev> wrote:
>
> > +1 (non-binding)
> >
> > Em seg., 21 de mar. de 2022 às 09:23, David Li <li...@apache.org>
> > escreveu:
> >
> > > My vote: +1
> > >
> > > Are any PMC members available to take a look? (Also note, anyone can
> > still
> > > cast non-binding votes.)
> > >
> > > -David
> > >
> > > On Wed, Mar 16, 2022, at 15:50, David Li wrote:
> > > > Hello,
> > > >
> > > > Jose Almeida and James Duong have proposed two additions to Arrow
> > > > Flight SQL, an experimental protocol for interacting with SQL
> databases
> > > > over Arrow Flight. The purpose of these additions is to provide
> > > > necessary metadata for implementing a JDBC driver on top of Flight
> SQL
> > > > [1].
> > > >
> > > > The additions are as follows:
> > > >
> > > > - As part of returned schemas, include metadata about the underlying
> > > > SQL data type [2].
> > > > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> > > > supported SQL data types [3].
> > > >
> > > > Both pull requests implement the additions in C++ and Java and
> contain
> > > > integration tests.
> > > >
> > > > Please vote whether to accept these enhancements. The vote will be
> open
> > > > for at least 72 hours.
> > > >
> > > > [ ] +1 Accept these protocol additions
> > > > [ ] +0
> > > > [ ] -1 Do not accept these protocol additions because…
> > > >
> > > > [1]: https://github.com/apache/arrow/pull/12254
> > > > [2]: https://github.com/apache/arrow/pull/11999
> > > > [3]: https://github.com/apache/arrow/pull/11982
> > > >
> > > > -David
> > >
> >
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Iury Salino <iu...@dev.simbioseventures.com>.
+1 (non-binding)
On Mon, Mar 21, 2022 at 2:54 PM Vinicius Fraga <sx...@gmail.com> wrote:
> +1 (non-binding)
>
> On Mon, 21 Mar 2022 at 14:53, Rafael Telles <ra...@telles.dev> wrote:
>
> > +1 (non-binding)
> >
> > Em seg., 21 de mar. de 2022 às 09:23, David Li <li...@apache.org>
> > escreveu:
> >
> > > My vote: +1
> > >
> > > Are any PMC members available to take a look? (Also note, anyone can
> > still
> > > cast non-binding votes.)
> > >
> > > -David
> > >
> > > On Wed, Mar 16, 2022, at 15:50, David Li wrote:
> > > > Hello,
> > > >
> > > > Jose Almeida and James Duong have proposed two additions to Arrow
> > > > Flight SQL, an experimental protocol for interacting with SQL
> databases
> > > > over Arrow Flight. The purpose of these additions is to provide
> > > > necessary metadata for implementing a JDBC driver on top of Flight
> SQL
> > > > [1].
> > > >
> > > > The additions are as follows:
> > > >
> > > > - As part of returned schemas, include metadata about the underlying
> > > > SQL data type [2].
> > > > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> > > > supported SQL data types [3].
> > > >
> > > > Both pull requests implement the additions in C++ and Java and
> contain
> > > > integration tests.
> > > >
> > > > Please vote whether to accept these enhancements. The vote will be
> open
> > > > for at least 72 hours.
> > > >
> > > > [ ] +1 Accept these protocol additions
> > > > [ ] +0
> > > > [ ] -1 Do not accept these protocol additions because…
> > > >
> > > > [1]: https://github.com/apache/arrow/pull/12254
> > > > [2]: https://github.com/apache/arrow/pull/11999
> > > > [3]: https://github.com/apache/arrow/pull/11982
> > > >
> > > > -David
> > >
> >
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Vinicius Fraga <sx...@gmail.com>.
+1 (non-binding)
On Mon, 21 Mar 2022 at 14:53, Rafael Telles <ra...@telles.dev> wrote:
> +1 (non-binding)
>
> Em seg., 21 de mar. de 2022 às 09:23, David Li <li...@apache.org>
> escreveu:
>
> > My vote: +1
> >
> > Are any PMC members available to take a look? (Also note, anyone can
> still
> > cast non-binding votes.)
> >
> > -David
> >
> > On Wed, Mar 16, 2022, at 15:50, David Li wrote:
> > > Hello,
> > >
> > > Jose Almeida and James Duong have proposed two additions to Arrow
> > > Flight SQL, an experimental protocol for interacting with SQL databases
> > > over Arrow Flight. The purpose of these additions is to provide
> > > necessary metadata for implementing a JDBC driver on top of Flight SQL
> > > [1].
> > >
> > > The additions are as follows:
> > >
> > > - As part of returned schemas, include metadata about the underlying
> > > SQL data type [2].
> > > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> > > supported SQL data types [3].
> > >
> > > Both pull requests implement the additions in C++ and Java and contain
> > > integration tests.
> > >
> > > Please vote whether to accept these enhancements. The vote will be open
> > > for at least 72 hours.
> > >
> > > [ ] +1 Accept these protocol additions
> > > [ ] +0
> > > [ ] -1 Do not accept these protocol additions because…
> > >
> > > [1]: https://github.com/apache/arrow/pull/12254
> > > [2]: https://github.com/apache/arrow/pull/11999
> > > [3]: https://github.com/apache/arrow/pull/11982
> > >
> > > -David
> >
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by Rafael Telles <ra...@telles.dev>.
+1 (non-binding)
Em seg., 21 de mar. de 2022 às 09:23, David Li <li...@apache.org>
escreveu:
> My vote: +1
>
> Are any PMC members available to take a look? (Also note, anyone can still
> cast non-binding votes.)
>
> -David
>
> On Wed, Mar 16, 2022, at 15:50, David Li wrote:
> > Hello,
> >
> > Jose Almeida and James Duong have proposed two additions to Arrow
> > Flight SQL, an experimental protocol for interacting with SQL databases
> > over Arrow Flight. The purpose of these additions is to provide
> > necessary metadata for implementing a JDBC driver on top of Flight SQL
> > [1].
> >
> > The additions are as follows:
> >
> > - As part of returned schemas, include metadata about the underlying
> > SQL data type [2].
> > - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> > supported SQL data types [3].
> >
> > Both pull requests implement the additions in C++ and Java and contain
> > integration tests.
> >
> > Please vote whether to accept these enhancements. The vote will be open
> > for at least 72 hours.
> >
> > [ ] +1 Accept these protocol additions
> > [ ] +0
> > [ ] -1 Do not accept these protocol additions because…
> >
> > [1]: https://github.com/apache/arrow/pull/12254
> > [2]: https://github.com/apache/arrow/pull/11999
> > [3]: https://github.com/apache/arrow/pull/11982
> >
> > -David
>
Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas
Posted by David Li <li...@apache.org>.
My vote: +1
Are any PMC members available to take a look? (Also note, anyone can still cast non-binding votes.)
-David
On Wed, Mar 16, 2022, at 15:50, David Li wrote:
> Hello,
>
> Jose Almeida and James Duong have proposed two additions to Arrow
> Flight SQL, an experimental protocol for interacting with SQL databases
> over Arrow Flight. The purpose of these additions is to provide
> necessary metadata for implementing a JDBC driver on top of Flight SQL
> [1].
>
> The additions are as follows:
>
> - As part of returned schemas, include metadata about the underlying
> SQL data type [2].
> - Add a new RPC endpoint, GetXdbcTypeInfo, to get metadata about the
> supported SQL data types [3].
>
> Both pull requests implement the additions in C++ and Java and contain
> integration tests.
>
> Please vote whether to accept these enhancements. The vote will be open
> for at least 72 hours.
>
> [ ] +1 Accept these protocol additions
> [ ] +0
> [ ] -1 Do not accept these protocol additions because…
>
> [1]: https://github.com/apache/arrow/pull/12254
> [2]: https://github.com/apache/arrow/pull/11999
> [3]: https://github.com/apache/arrow/pull/11982
>
> -David