You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by David Li <li...@apache.org> on 2022/08/31 23:51:07 UTC
[VOTE] Substrait for Flight SQL
Hello,
I am proposing to extend the Flight SQL specification with the following features:
- Support for queries using Substrait [1]
- Explicit transaction RPCs
- Explicit cancellation of distributed queries
The proposal was discussed previously in [2]. The new Protobuf definitions, an implementation for C++ and Java, and integration tests, are available at [3] with some further discussion.
The vote will be open for at least 72 hours.
[ ] +1 : Accept the proposal
[ ] 0 : No opinion
[ ] -1 : Do not accept this proposal because...
[1]: https://substrait.io/
[2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
[3]: https://github.com/apache/arrow/pull/13492
Re: [VOTE] Substrait for Flight SQL
Posted by James Henderson <jm...@juxt.pro>.
+1 (non-binding)
--
James Henderson
XTDB Development Manager at JUXT
Email jms@juxt.pro
Website https://juxt.pro
[image: photo]
Re: [VOTE] Substrait for Flight SQL
Posted by James Duong <ja...@bitquilltech.com.INVALID>.
+1 (non-binding)
On Wed, Aug 31, 2022 at 9:49 PM Jacques Nadeau <ja...@apache.org> wrote:
> +1 (binding)
>
> On Wed, Aug 31, 2022, 5:15 PM Larry White <lj...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Wed, Aug 31, 2022 at 7:55 PM Vinicius Fraga <sx...@gmail.com>
> wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Wed, 31 Aug 2022, 20:51 David Li, <li...@apache.org> wrote:
> > >
> > > > Hello,
> > > >
> > > > I am proposing to extend the Flight SQL specification with the
> > following
> > > > features:
> > > >
> > > > - Support for queries using Substrait [1]
> > > > - Explicit transaction RPCs
> > > > - Explicit cancellation of distributed queries
> > > >
> > > > The proposal was discussed previously in [2]. The new Protobuf
> > > > definitions, an implementation for C++ and Java, and integration
> tests,
> > > are
> > > > available at [3] with some further discussion.
> > > >
> > > > The vote will be open for at least 72 hours.
> > > >
> > > > [ ] +1 : Accept the proposal
> > > > [ ] 0 : No opinion
> > > > [ ] -1 : Do not accept this proposal because...
> > > >
> > > > [1]: https://substrait.io/
> > > > [2]:
> https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
> > > > [3]: https://github.com/apache/arrow/pull/13492
> > > >
> > >
> >
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: [VOTE] Substrait for Flight SQL
Posted by Jacques Nadeau <ja...@apache.org>.
+1 (binding)
On Wed, Aug 31, 2022, 5:15 PM Larry White <lj...@gmail.com> wrote:
> +1 (non-binding)
>
> On Wed, Aug 31, 2022 at 7:55 PM Vinicius Fraga <sx...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Wed, 31 Aug 2022, 20:51 David Li, <li...@apache.org> wrote:
> >
> > > Hello,
> > >
> > > I am proposing to extend the Flight SQL specification with the
> following
> > > features:
> > >
> > > - Support for queries using Substrait [1]
> > > - Explicit transaction RPCs
> > > - Explicit cancellation of distributed queries
> > >
> > > The proposal was discussed previously in [2]. The new Protobuf
> > > definitions, an implementation for C++ and Java, and integration tests,
> > are
> > > available at [3] with some further discussion.
> > >
> > > The vote will be open for at least 72 hours.
> > >
> > > [ ] +1 : Accept the proposal
> > > [ ] 0 : No opinion
> > > [ ] -1 : Do not accept this proposal because...
> > >
> > > [1]: https://substrait.io/
> > > [2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
> > > [3]: https://github.com/apache/arrow/pull/13492
> > >
> >
>
Re: [VOTE] Substrait for Flight SQL
Posted by Larry White <lj...@gmail.com>.
+1 (non-binding)
On Wed, Aug 31, 2022 at 7:55 PM Vinicius Fraga <sx...@gmail.com> wrote:
> +1 (non-binding)
>
> On Wed, 31 Aug 2022, 20:51 David Li, <li...@apache.org> wrote:
>
> > Hello,
> >
> > I am proposing to extend the Flight SQL specification with the following
> > features:
> >
> > - Support for queries using Substrait [1]
> > - Explicit transaction RPCs
> > - Explicit cancellation of distributed queries
> >
> > The proposal was discussed previously in [2]. The new Protobuf
> > definitions, an implementation for C++ and Java, and integration tests,
> are
> > available at [3] with some further discussion.
> >
> > The vote will be open for at least 72 hours.
> >
> > [ ] +1 : Accept the proposal
> > [ ] 0 : No opinion
> > [ ] -1 : Do not accept this proposal because...
> >
> > [1]: https://substrait.io/
> > [2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
> > [3]: https://github.com/apache/arrow/pull/13492
> >
>
Re: [VOTE] Substrait for Flight SQL
Posted by Vinicius Fraga <sx...@gmail.com>.
+1 (non-binding)
On Wed, 31 Aug 2022, 20:51 David Li, <li...@apache.org> wrote:
> Hello,
>
> I am proposing to extend the Flight SQL specification with the following
> features:
>
> - Support for queries using Substrait [1]
> - Explicit transaction RPCs
> - Explicit cancellation of distributed queries
>
> The proposal was discussed previously in [2]. The new Protobuf
> definitions, an implementation for C++ and Java, and integration tests, are
> available at [3] with some further discussion.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 : Accept the proposal
> [ ] 0 : No opinion
> [ ] -1 : Do not accept this proposal because...
>
> [1]: https://substrait.io/
> [2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
> [3]: https://github.com/apache/arrow/pull/13492
>
Re: [VOTE] Substrait for Flight SQL
Posted by Gavin Ray <ra...@gmail.com>.
Hooray!
On Fri, Sep 16, 2022 at 11:08 AM David Li <li...@apache.org> wrote:
> The PR is now merged:
> https://github.com/apache/arrow/commit/3ce40143f8a836df058ec5fe1b29d9da5ede169d
>
> Thanks all!
>
> On Sat, Sep 10, 2022, at 18:15, David Li wrote:
> > The vote passes with 5 binding votes and 7 non-binding votes. Thanks all!
> >
> > I will rebase the PR and ensure CI passes before merging.
> >
> > On Fri, Sep 9, 2022, at 16:14, Wes McKinney wrote:
> >> +1 (binding)
> >>
> >> On Thu, Sep 8, 2022 at 9:12 PM Jacques Nadeau <ja...@apache.org>
> wrote:
> >>>
> >>> My vote continues to be +1
> >>>
> >>> On Thu, Sep 8, 2022 at 11:44 AM Neal Richardson <
> neal.p.richardson@gmail.com>
> >>> wrote:
> >>>
> >>> > +1
> >>> >
> >>> > Neal
> >>> >
> >>> > On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com>
> wrote:
> >>> >
> >>> > > +1 (non-binding)
> >>> > >
> >>> > > On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com>
> wrote:
> >>> > >
> >>> > > > Oh, so that's what "non-binding" means in vote threads
> >>> > > > Those threads make a lot more sense now, thanks for the heads-up
> =)
> >>> > > >
> >>> > > > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org>
> wrote:
> >>> > > >
> >>> > > > > Non-binding votes are always welcome and encouraged! Was just
> trying
> >>> > to
> >>> > > > > make sure we have the minimum 3 binding votes here but it
> turns out I
> >>> > > > can't
> >>> > > > > count and I make three.
> >>> > > > >
> >>> > > > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> >>> > > > > > If non-PMC can vote, I'll also give a huge +1
> >>> > > > > >
> >>> > > > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> >>> > > > > <ma...@voltrondata.com.invalid>
> >>> > > > > > wrote:
> >>> > > > > >
> >>> > > > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like
> the idea
> >>> > > of
> >>> > > > > >> integrating Substrait plans into Flight SQL if possible and
> it
> >>> > > aligns
> >>> > > > > >> with the arrow-adbc work.
> >>> > > > > >>
> >>> > > > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
> >>> > > > lidavidm@apache.org>
> >>> > > > > >> wrote:
> >>> > > > > >> > My vote: +1 (binding)
> >>> > > > > >> >
> >>> > > > > >> > Are any other PMC members available to take a look?
> >>> > > > > >> >
> >>> > > > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> >>> > > > > >> >> Fair enough. For the record, my main concern with ad-hoc
> >>> > > > conventions
> >>> > > > > >> >> such as "number of milliseconds expressed as an
> integer" is
> >>> > the
> >>> > > > poor
> >>> > > > > >> >> usability and the potential for confusion (not to
> mention that
> >>> > > > > >> >> sometimes
> >>> > > > > >> >> the need for a higher precision can lead to add another
> set of
> >>> > > > > >> >> APIs, but
> >>> > > > > >> >> that's unlikely to be the case here :-)).
> >>> > > > > >> >>
> >>> > > > > >> >> Regards
> >>> > > > > >> >>
> >>> > > > > >> >> Antoine.
> >>> > > > > >> >>
> >>> > > > > >> >>
> >>> > > > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> >>> > > > > >> >>> Absent further comments on this I would rather avoid
> adding a
> >>> > > > > >> >>> potentially breaking (even if likely compatible) change
> to the
> >>> > > > > >> >>> schema of this endpoint, if that's acceptable. I don't
> think a
> >>> > > > > >> >>> millisecond timeout is all too different from
> floating-point
> >>> > > > > >> >>> seconds (especially at the scale of network RPCs).
> >>> > > > > >> >>>
> >>> > > > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> >>> > > > > >> >>>> We could add a new type code to the union. Presumably
> >>> > > consumers
> >>> > > > > >> >>>> would
> >>> > > > > >> >>>> just error on or ignore such values (the libraries
> just hand
> >>> > > the
> >>> > > > > >> >>>> Arrow
> >>> > > > > >> >>>> array to the application, so it's up to the
> application what
> >>> > > to
> >>> > > > > >> >>>> do with
> >>> > > > > >> >>>> an unknown type code). (And for a new consumer
> talking to an
> >>> > > old
> >>> > > > > >> >>>> server, the new type code would just never come up,
> so the
> >>> > > only
> >>> > > > > >> >>>> issue
> >>> > > > > >> >>>> would be if it strictly validates the returned
> schema.)
> >>> > > > > >> >>>>
> >>> > > > > >> >>>> If there's support, I can make this revision as well.
> >>> > > > > >> >>>>
> >>> > > > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> >>> > > > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> >>> > > > > >> >>>>>> Thanks Antoine!
> >>> > > > > >> >>>>>>
> >>> > > > > >> >>>>>> I've updated the PR (except for the comment about
> timeout
> >>> > > > > >> >>>>>> units, since SqlInfo values can't be doubles/floats
> unless
> >>> > we
> >>> > > > > >> >>>>>> change the schema there)
> >>> > > > > >> >>>>>
> >>> > > > > >> >>>>> Can we change the schema in a backwards-compatible
> way?
> >>> > > > > >>
> >>> > > > > >>
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > thanks
> >>> > > ashish
> >>> > >
> >>> >
>
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
The PR is now merged: https://github.com/apache/arrow/commit/3ce40143f8a836df058ec5fe1b29d9da5ede169d
Thanks all!
On Sat, Sep 10, 2022, at 18:15, David Li wrote:
> The vote passes with 5 binding votes and 7 non-binding votes. Thanks all!
>
> I will rebase the PR and ensure CI passes before merging.
>
> On Fri, Sep 9, 2022, at 16:14, Wes McKinney wrote:
>> +1 (binding)
>>
>> On Thu, Sep 8, 2022 at 9:12 PM Jacques Nadeau <ja...@apache.org> wrote:
>>>
>>> My vote continues to be +1
>>>
>>> On Thu, Sep 8, 2022 at 11:44 AM Neal Richardson <ne...@gmail.com>
>>> wrote:
>>>
>>> > +1
>>> >
>>> > Neal
>>> >
>>> > On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com> wrote:
>>> >
>>> > > +1 (non-binding)
>>> > >
>>> > > On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
>>> > >
>>> > > > Oh, so that's what "non-binding" means in vote threads
>>> > > > Those threads make a lot more sense now, thanks for the heads-up =)
>>> > > >
>>> > > > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
>>> > > >
>>> > > > > Non-binding votes are always welcome and encouraged! Was just trying
>>> > to
>>> > > > > make sure we have the minimum 3 binding votes here but it turns out I
>>> > > > can't
>>> > > > > count and I make three.
>>> > > > >
>>> > > > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
>>> > > > > > If non-PMC can vote, I'll also give a huge +1
>>> > > > > >
>>> > > > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
>>> > > > > <ma...@voltrondata.com.invalid>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea
>>> > > of
>>> > > > > >> integrating Substrait plans into Flight SQL if possible and it
>>> > > aligns
>>> > > > > >> with the arrow-adbc work.
>>> > > > > >>
>>> > > > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
>>> > > > lidavidm@apache.org>
>>> > > > > >> wrote:
>>> > > > > >> > My vote: +1 (binding)
>>> > > > > >> >
>>> > > > > >> > Are any other PMC members available to take a look?
>>> > > > > >> >
>>> > > > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
>>> > > > > >> >> Fair enough. For the record, my main concern with ad-hoc
>>> > > > conventions
>>> > > > > >> >> such as "number of milliseconds expressed as an integer" is
>>> > the
>>> > > > poor
>>> > > > > >> >> usability and the potential for confusion (not to mention that
>>> > > > > >> >> sometimes
>>> > > > > >> >> the need for a higher precision can lead to add another set of
>>> > > > > >> >> APIs, but
>>> > > > > >> >> that's unlikely to be the case here :-)).
>>> > > > > >> >>
>>> > > > > >> >> Regards
>>> > > > > >> >>
>>> > > > > >> >> Antoine.
>>> > > > > >> >>
>>> > > > > >> >>
>>> > > > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
>>> > > > > >> >>> Absent further comments on this I would rather avoid adding a
>>> > > > > >> >>> potentially breaking (even if likely compatible) change to the
>>> > > > > >> >>> schema of this endpoint, if that's acceptable. I don't think a
>>> > > > > >> >>> millisecond timeout is all too different from floating-point
>>> > > > > >> >>> seconds (especially at the scale of network RPCs).
>>> > > > > >> >>>
>>> > > > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>>> > > > > >> >>>> We could add a new type code to the union. Presumably
>>> > > consumers
>>> > > > > >> >>>> would
>>> > > > > >> >>>> just error on or ignore such values (the libraries just hand
>>> > > the
>>> > > > > >> >>>> Arrow
>>> > > > > >> >>>> array to the application, so it's up to the application what
>>> > > to
>>> > > > > >> >>>> do with
>>> > > > > >> >>>> an unknown type code). (And for a new consumer talking to an
>>> > > old
>>> > > > > >> >>>> server, the new type code would just never come up, so the
>>> > > only
>>> > > > > >> >>>> issue
>>> > > > > >> >>>> would be if it strictly validates the returned schema.)
>>> > > > > >> >>>>
>>> > > > > >> >>>> If there's support, I can make this revision as well.
>>> > > > > >> >>>>
>>> > > > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>>> > > > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
>>> > > > > >> >>>>>> Thanks Antoine!
>>> > > > > >> >>>>>>
>>> > > > > >> >>>>>> I've updated the PR (except for the comment about timeout
>>> > > > > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless
>>> > we
>>> > > > > >> >>>>>> change the schema there)
>>> > > > > >> >>>>>
>>> > > > > >> >>>>> Can we change the schema in a backwards-compatible way?
>>> > > > > >>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > > thanks
>>> > > ashish
>>> > >
>>> >
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
The vote passes with 5 binding votes and 7 non-binding votes. Thanks all!
I will rebase the PR and ensure CI passes before merging.
On Fri, Sep 9, 2022, at 16:14, Wes McKinney wrote:
> +1 (binding)
>
> On Thu, Sep 8, 2022 at 9:12 PM Jacques Nadeau <ja...@apache.org> wrote:
>>
>> My vote continues to be +1
>>
>> On Thu, Sep 8, 2022 at 11:44 AM Neal Richardson <ne...@gmail.com>
>> wrote:
>>
>> > +1
>> >
>> > Neal
>> >
>> > On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com> wrote:
>> >
>> > > +1 (non-binding)
>> > >
>> > > On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
>> > >
>> > > > Oh, so that's what "non-binding" means in vote threads
>> > > > Those threads make a lot more sense now, thanks for the heads-up =)
>> > > >
>> > > > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
>> > > >
>> > > > > Non-binding votes are always welcome and encouraged! Was just trying
>> > to
>> > > > > make sure we have the minimum 3 binding votes here but it turns out I
>> > > > can't
>> > > > > count and I make three.
>> > > > >
>> > > > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
>> > > > > > If non-PMC can vote, I'll also give a huge +1
>> > > > > >
>> > > > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
>> > > > > <ma...@voltrondata.com.invalid>
>> > > > > > wrote:
>> > > > > >
>> > > > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea
>> > > of
>> > > > > >> integrating Substrait plans into Flight SQL if possible and it
>> > > aligns
>> > > > > >> with the arrow-adbc work.
>> > > > > >>
>> > > > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
>> > > > lidavidm@apache.org>
>> > > > > >> wrote:
>> > > > > >> > My vote: +1 (binding)
>> > > > > >> >
>> > > > > >> > Are any other PMC members available to take a look?
>> > > > > >> >
>> > > > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
>> > > > > >> >> Fair enough. For the record, my main concern with ad-hoc
>> > > > conventions
>> > > > > >> >> such as "number of milliseconds expressed as an integer" is
>> > the
>> > > > poor
>> > > > > >> >> usability and the potential for confusion (not to mention that
>> > > > > >> >> sometimes
>> > > > > >> >> the need for a higher precision can lead to add another set of
>> > > > > >> >> APIs, but
>> > > > > >> >> that's unlikely to be the case here :-)).
>> > > > > >> >>
>> > > > > >> >> Regards
>> > > > > >> >>
>> > > > > >> >> Antoine.
>> > > > > >> >>
>> > > > > >> >>
>> > > > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
>> > > > > >> >>> Absent further comments on this I would rather avoid adding a
>> > > > > >> >>> potentially breaking (even if likely compatible) change to the
>> > > > > >> >>> schema of this endpoint, if that's acceptable. I don't think a
>> > > > > >> >>> millisecond timeout is all too different from floating-point
>> > > > > >> >>> seconds (especially at the scale of network RPCs).
>> > > > > >> >>>
>> > > > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>> > > > > >> >>>> We could add a new type code to the union. Presumably
>> > > consumers
>> > > > > >> >>>> would
>> > > > > >> >>>> just error on or ignore such values (the libraries just hand
>> > > the
>> > > > > >> >>>> Arrow
>> > > > > >> >>>> array to the application, so it's up to the application what
>> > > to
>> > > > > >> >>>> do with
>> > > > > >> >>>> an unknown type code). (And for a new consumer talking to an
>> > > old
>> > > > > >> >>>> server, the new type code would just never come up, so the
>> > > only
>> > > > > >> >>>> issue
>> > > > > >> >>>> would be if it strictly validates the returned schema.)
>> > > > > >> >>>>
>> > > > > >> >>>> If there's support, I can make this revision as well.
>> > > > > >> >>>>
>> > > > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>> > > > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
>> > > > > >> >>>>>> Thanks Antoine!
>> > > > > >> >>>>>>
>> > > > > >> >>>>>> I've updated the PR (except for the comment about timeout
>> > > > > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless
>> > we
>> > > > > >> >>>>>> change the schema there)
>> > > > > >> >>>>>
>> > > > > >> >>>>> Can we change the schema in a backwards-compatible way?
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > thanks
>> > > ashish
>> > >
>> >
Re: [VOTE] Substrait for Flight SQL
Posted by Wes McKinney <we...@gmail.com>.
+1 (binding)
On Thu, Sep 8, 2022 at 9:12 PM Jacques Nadeau <ja...@apache.org> wrote:
>
> My vote continues to be +1
>
> On Thu, Sep 8, 2022 at 11:44 AM Neal Richardson <ne...@gmail.com>
> wrote:
>
> > +1
> >
> > Neal
> >
> > On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com> wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
> > >
> > > > Oh, so that's what "non-binding" means in vote threads
> > > > Those threads make a lot more sense now, thanks for the heads-up =)
> > > >
> > > > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
> > > >
> > > > > Non-binding votes are always welcome and encouraged! Was just trying
> > to
> > > > > make sure we have the minimum 3 binding votes here but it turns out I
> > > > can't
> > > > > count and I make three.
> > > > >
> > > > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> > > > > > If non-PMC can vote, I'll also give a huge +1
> > > > > >
> > > > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> > > > > <ma...@voltrondata.com.invalid>
> > > > > > wrote:
> > > > > >
> > > > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea
> > > of
> > > > > >> integrating Substrait plans into Flight SQL if possible and it
> > > aligns
> > > > > >> with the arrow-adbc work.
> > > > > >>
> > > > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
> > > > lidavidm@apache.org>
> > > > > >> wrote:
> > > > > >> > My vote: +1 (binding)
> > > > > >> >
> > > > > >> > Are any other PMC members available to take a look?
> > > > > >> >
> > > > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> > > > > >> >> Fair enough. For the record, my main concern with ad-hoc
> > > > conventions
> > > > > >> >> such as "number of milliseconds expressed as an integer" is
> > the
> > > > poor
> > > > > >> >> usability and the potential for confusion (not to mention that
> > > > > >> >> sometimes
> > > > > >> >> the need for a higher precision can lead to add another set of
> > > > > >> >> APIs, but
> > > > > >> >> that's unlikely to be the case here :-)).
> > > > > >> >>
> > > > > >> >> Regards
> > > > > >> >>
> > > > > >> >> Antoine.
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> > > > > >> >>> Absent further comments on this I would rather avoid adding a
> > > > > >> >>> potentially breaking (even if likely compatible) change to the
> > > > > >> >>> schema of this endpoint, if that's acceptable. I don't think a
> > > > > >> >>> millisecond timeout is all too different from floating-point
> > > > > >> >>> seconds (especially at the scale of network RPCs).
> > > > > >> >>>
> > > > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> > > > > >> >>>> We could add a new type code to the union. Presumably
> > > consumers
> > > > > >> >>>> would
> > > > > >> >>>> just error on or ignore such values (the libraries just hand
> > > the
> > > > > >> >>>> Arrow
> > > > > >> >>>> array to the application, so it's up to the application what
> > > to
> > > > > >> >>>> do with
> > > > > >> >>>> an unknown type code). (And for a new consumer talking to an
> > > old
> > > > > >> >>>> server, the new type code would just never come up, so the
> > > only
> > > > > >> >>>> issue
> > > > > >> >>>> would be if it strictly validates the returned schema.)
> > > > > >> >>>>
> > > > > >> >>>> If there's support, I can make this revision as well.
> > > > > >> >>>>
> > > > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> > > > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> > > > > >> >>>>>> Thanks Antoine!
> > > > > >> >>>>>>
> > > > > >> >>>>>> I've updated the PR (except for the comment about timeout
> > > > > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless
> > we
> > > > > >> >>>>>> change the schema there)
> > > > > >> >>>>>
> > > > > >> >>>>> Can we change the schema in a backwards-compatible way?
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > > thanks
> > > ashish
> > >
> >
Re: [VOTE] Substrait for Flight SQL
Posted by Jacques Nadeau <ja...@apache.org>.
My vote continues to be +1
On Thu, Sep 8, 2022 at 11:44 AM Neal Richardson <ne...@gmail.com>
wrote:
> +1
>
> Neal
>
> On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
> >
> > > Oh, so that's what "non-binding" means in vote threads
> > > Those threads make a lot more sense now, thanks for the heads-up =)
> > >
> > > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
> > >
> > > > Non-binding votes are always welcome and encouraged! Was just trying
> to
> > > > make sure we have the minimum 3 binding votes here but it turns out I
> > > can't
> > > > count and I make three.
> > > >
> > > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> > > > > If non-PMC can vote, I'll also give a huge +1
> > > > >
> > > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> > > > <ma...@voltrondata.com.invalid>
> > > > > wrote:
> > > > >
> > > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea
> > of
> > > > >> integrating Substrait plans into Flight SQL if possible and it
> > aligns
> > > > >> with the arrow-adbc work.
> > > > >>
> > > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
> > > lidavidm@apache.org>
> > > > >> wrote:
> > > > >> > My vote: +1 (binding)
> > > > >> >
> > > > >> > Are any other PMC members available to take a look?
> > > > >> >
> > > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> > > > >> >> Fair enough. For the record, my main concern with ad-hoc
> > > conventions
> > > > >> >> such as "number of milliseconds expressed as an integer" is
> the
> > > poor
> > > > >> >> usability and the potential for confusion (not to mention that
> > > > >> >> sometimes
> > > > >> >> the need for a higher precision can lead to add another set of
> > > > >> >> APIs, but
> > > > >> >> that's unlikely to be the case here :-)).
> > > > >> >>
> > > > >> >> Regards
> > > > >> >>
> > > > >> >> Antoine.
> > > > >> >>
> > > > >> >>
> > > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> > > > >> >>> Absent further comments on this I would rather avoid adding a
> > > > >> >>> potentially breaking (even if likely compatible) change to the
> > > > >> >>> schema of this endpoint, if that's acceptable. I don't think a
> > > > >> >>> millisecond timeout is all too different from floating-point
> > > > >> >>> seconds (especially at the scale of network RPCs).
> > > > >> >>>
> > > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> > > > >> >>>> We could add a new type code to the union. Presumably
> > consumers
> > > > >> >>>> would
> > > > >> >>>> just error on or ignore such values (the libraries just hand
> > the
> > > > >> >>>> Arrow
> > > > >> >>>> array to the application, so it's up to the application what
> > to
> > > > >> >>>> do with
> > > > >> >>>> an unknown type code). (And for a new consumer talking to an
> > old
> > > > >> >>>> server, the new type code would just never come up, so the
> > only
> > > > >> >>>> issue
> > > > >> >>>> would be if it strictly validates the returned schema.)
> > > > >> >>>>
> > > > >> >>>> If there's support, I can make this revision as well.
> > > > >> >>>>
> > > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> > > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> > > > >> >>>>>> Thanks Antoine!
> > > > >> >>>>>>
> > > > >> >>>>>> I've updated the PR (except for the comment about timeout
> > > > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless
> we
> > > > >> >>>>>> change the schema there)
> > > > >> >>>>>
> > > > >> >>>>> Can we change the schema in a backwards-compatible way?
> > > > >>
> > > > >>
> > > >
> > >
> >
> >
> > --
> > thanks
> > ashish
> >
>
Re: [VOTE] Substrait for Flight SQL
Posted by Neal Richardson <ne...@gmail.com>.
+1
Neal
On Thu, Sep 8, 2022 at 2:15 PM Ashish <pa...@gmail.com> wrote:
> +1 (non-binding)
>
> On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
>
> > Oh, so that's what "non-binding" means in vote threads
> > Those threads make a lot more sense now, thanks for the heads-up =)
> >
> > On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
> >
> > > Non-binding votes are always welcome and encouraged! Was just trying to
> > > make sure we have the minimum 3 binding votes here but it turns out I
> > can't
> > > count and I make three.
> > >
> > > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> > > > If non-PMC can vote, I'll also give a huge +1
> > > >
> > > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> > > <ma...@voltrondata.com.invalid>
> > > > wrote:
> > > >
> > > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea
> of
> > > >> integrating Substrait plans into Flight SQL if possible and it
> aligns
> > > >> with the arrow-adbc work.
> > > >>
> > > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
> > lidavidm@apache.org>
> > > >> wrote:
> > > >> > My vote: +1 (binding)
> > > >> >
> > > >> > Are any other PMC members available to take a look?
> > > >> >
> > > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> > > >> >> Fair enough. For the record, my main concern with ad-hoc
> > conventions
> > > >> >> such as "number of milliseconds expressed as an integer" is the
> > poor
> > > >> >> usability and the potential for confusion (not to mention that
> > > >> >> sometimes
> > > >> >> the need for a higher precision can lead to add another set of
> > > >> >> APIs, but
> > > >> >> that's unlikely to be the case here :-)).
> > > >> >>
> > > >> >> Regards
> > > >> >>
> > > >> >> Antoine.
> > > >> >>
> > > >> >>
> > > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> > > >> >>> Absent further comments on this I would rather avoid adding a
> > > >> >>> potentially breaking (even if likely compatible) change to the
> > > >> >>> schema of this endpoint, if that's acceptable. I don't think a
> > > >> >>> millisecond timeout is all too different from floating-point
> > > >> >>> seconds (especially at the scale of network RPCs).
> > > >> >>>
> > > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> > > >> >>>> We could add a new type code to the union. Presumably
> consumers
> > > >> >>>> would
> > > >> >>>> just error on or ignore such values (the libraries just hand
> the
> > > >> >>>> Arrow
> > > >> >>>> array to the application, so it's up to the application what
> to
> > > >> >>>> do with
> > > >> >>>> an unknown type code). (And for a new consumer talking to an
> old
> > > >> >>>> server, the new type code would just never come up, so the
> only
> > > >> >>>> issue
> > > >> >>>> would be if it strictly validates the returned schema.)
> > > >> >>>>
> > > >> >>>> If there's support, I can make this revision as well.
> > > >> >>>>
> > > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> > > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> > > >> >>>>>> Thanks Antoine!
> > > >> >>>>>>
> > > >> >>>>>> I've updated the PR (except for the comment about timeout
> > > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless we
> > > >> >>>>>> change the schema there)
> > > >> >>>>>
> > > >> >>>>> Can we change the schema in a backwards-compatible way?
> > > >>
> > > >>
> > >
> >
>
>
> --
> thanks
> ashish
>
Re: [VOTE] Substrait for Flight SQL
Posted by Ashish <pa...@gmail.com>.
+1 (non-binding)
On Thu, Sep 8, 2022 at 9:41 AM Gavin Ray <ra...@gmail.com> wrote:
> Oh, so that's what "non-binding" means in vote threads
> Those threads make a lot more sense now, thanks for the heads-up =)
>
> On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
>
> > Non-binding votes are always welcome and encouraged! Was just trying to
> > make sure we have the minimum 3 binding votes here but it turns out I
> can't
> > count and I make three.
> >
> > On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> > > If non-PMC can vote, I'll also give a huge +1
> > >
> > > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> > <ma...@voltrondata.com.invalid>
> > > wrote:
> > >
> > >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of
> > >> integrating Substrait plans into Flight SQL if possible and it aligns
> > >> with the arrow-adbc work.
> > >>
> > >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <
> lidavidm@apache.org>
> > >> wrote:
> > >> > My vote: +1 (binding)
> > >> >
> > >> > Are any other PMC members available to take a look?
> > >> >
> > >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> > >> >> Fair enough. For the record, my main concern with ad-hoc
> conventions
> > >> >> such as "number of milliseconds expressed as an integer" is the
> poor
> > >> >> usability and the potential for confusion (not to mention that
> > >> >> sometimes
> > >> >> the need for a higher precision can lead to add another set of
> > >> >> APIs, but
> > >> >> that's unlikely to be the case here :-)).
> > >> >>
> > >> >> Regards
> > >> >>
> > >> >> Antoine.
> > >> >>
> > >> >>
> > >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> > >> >>> Absent further comments on this I would rather avoid adding a
> > >> >>> potentially breaking (even if likely compatible) change to the
> > >> >>> schema of this endpoint, if that's acceptable. I don't think a
> > >> >>> millisecond timeout is all too different from floating-point
> > >> >>> seconds (especially at the scale of network RPCs).
> > >> >>>
> > >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> > >> >>>> We could add a new type code to the union. Presumably consumers
> > >> >>>> would
> > >> >>>> just error on or ignore such values (the libraries just hand the
> > >> >>>> Arrow
> > >> >>>> array to the application, so it's up to the application what to
> > >> >>>> do with
> > >> >>>> an unknown type code). (And for a new consumer talking to an old
> > >> >>>> server, the new type code would just never come up, so the only
> > >> >>>> issue
> > >> >>>> would be if it strictly validates the returned schema.)
> > >> >>>>
> > >> >>>> If there's support, I can make this revision as well.
> > >> >>>>
> > >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> > >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> > >> >>>>>> Thanks Antoine!
> > >> >>>>>>
> > >> >>>>>> I've updated the PR (except for the comment about timeout
> > >> >>>>>> units, since SqlInfo values can't be doubles/floats unless we
> > >> >>>>>> change the schema there)
> > >> >>>>>
> > >> >>>>> Can we change the schema in a backwards-compatible way?
> > >>
> > >>
> >
>
--
thanks
ashish
Re: [VOTE] Substrait for Flight SQL
Posted by Gavin Ray <ra...@gmail.com>.
Oh, so that's what "non-binding" means in vote threads
Those threads make a lot more sense now, thanks for the heads-up =)
On Thu, Sep 8, 2022 at 12:31 PM David Li <li...@apache.org> wrote:
> Non-binding votes are always welcome and encouraged! Was just trying to
> make sure we have the minimum 3 binding votes here but it turns out I can't
> count and I make three.
>
> On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> > If non-PMC can vote, I'll also give a huge +1
> >
> > On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol
> <ma...@voltrondata.com.invalid>
> > wrote:
> >
> >> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of
> >> integrating Substrait plans into Flight SQL if possible and it aligns
> >> with the arrow-adbc work.
> >>
> >> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <li...@apache.org>
> >> wrote:
> >> > My vote: +1 (binding)
> >> >
> >> > Are any other PMC members available to take a look?
> >> >
> >> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> >> >> Fair enough. For the record, my main concern with ad-hoc conventions
> >> >> such as "number of milliseconds expressed as an integer" is the poor
> >> >> usability and the potential for confusion (not to mention that
> >> >> sometimes
> >> >> the need for a higher precision can lead to add another set of
> >> >> APIs, but
> >> >> that's unlikely to be the case here :-)).
> >> >>
> >> >> Regards
> >> >>
> >> >> Antoine.
> >> >>
> >> >>
> >> >> Le 07/09/2022 à 14:21, David Li a écrit :
> >> >>> Absent further comments on this I would rather avoid adding a
> >> >>> potentially breaking (even if likely compatible) change to the
> >> >>> schema of this endpoint, if that's acceptable. I don't think a
> >> >>> millisecond timeout is all too different from floating-point
> >> >>> seconds (especially at the scale of network RPCs).
> >> >>>
> >> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> >> >>>> We could add a new type code to the union. Presumably consumers
> >> >>>> would
> >> >>>> just error on or ignore such values (the libraries just hand the
> >> >>>> Arrow
> >> >>>> array to the application, so it's up to the application what to
> >> >>>> do with
> >> >>>> an unknown type code). (And for a new consumer talking to an old
> >> >>>> server, the new type code would just never come up, so the only
> >> >>>> issue
> >> >>>> would be if it strictly validates the returned schema.)
> >> >>>>
> >> >>>> If there's support, I can make this revision as well.
> >> >>>>
> >> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> >> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> >> >>>>>> Thanks Antoine!
> >> >>>>>>
> >> >>>>>> I've updated the PR (except for the comment about timeout
> >> >>>>>> units, since SqlInfo values can't be doubles/floats unless we
> >> >>>>>> change the schema there)
> >> >>>>>
> >> >>>>> Can we change the schema in a backwards-compatible way?
> >>
> >>
>
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
Non-binding votes are always welcome and encouraged! Was just trying to make sure we have the minimum 3 binding votes here but it turns out I can't count and I make three.
On Thu, Sep 8, 2022, at 12:14, Gavin Ray wrote:
> If non-PMC can vote, I'll also give a huge +1
>
> On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol <ma...@voltrondata.com.invalid>
> wrote:
>
>> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of
>> integrating Substrait plans into Flight SQL if possible and it aligns
>> with the arrow-adbc work.
>>
>> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <li...@apache.org>
>> wrote:
>> > My vote: +1 (binding)
>> >
>> > Are any other PMC members available to take a look?
>> >
>> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
>> >> Fair enough. For the record, my main concern with ad-hoc conventions
>> >> such as "number of milliseconds expressed as an integer" is the poor
>> >> usability and the potential for confusion (not to mention that
>> >> sometimes
>> >> the need for a higher precision can lead to add another set of
>> >> APIs, but
>> >> that's unlikely to be the case here :-)).
>> >>
>> >> Regards
>> >>
>> >> Antoine.
>> >>
>> >>
>> >> Le 07/09/2022 à 14:21, David Li a écrit :
>> >>> Absent further comments on this I would rather avoid adding a
>> >>> potentially breaking (even if likely compatible) change to the
>> >>> schema of this endpoint, if that's acceptable. I don't think a
>> >>> millisecond timeout is all too different from floating-point
>> >>> seconds (especially at the scale of network RPCs).
>> >>>
>> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>> >>>> We could add a new type code to the union. Presumably consumers
>> >>>> would
>> >>>> just error on or ignore such values (the libraries just hand the
>> >>>> Arrow
>> >>>> array to the application, so it's up to the application what to
>> >>>> do with
>> >>>> an unknown type code). (And for a new consumer talking to an old
>> >>>> server, the new type code would just never come up, so the only
>> >>>> issue
>> >>>> would be if it strictly validates the returned schema.)
>> >>>>
>> >>>> If there's support, I can make this revision as well.
>> >>>>
>> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
>> >>>>>> Thanks Antoine!
>> >>>>>>
>> >>>>>> I've updated the PR (except for the comment about timeout
>> >>>>>> units, since SqlInfo values can't be doubles/floats unless we
>> >>>>>> change the schema there)
>> >>>>>
>> >>>>> Can we change the schema in a backwards-compatible way?
>>
>>
Re: [VOTE] Substrait for Flight SQL
Posted by Gavin Ray <ra...@gmail.com>.
If non-PMC can vote, I'll also give a huge +1
On Thu, Sep 8, 2022 at 11:34 AM Matthew Topol <ma...@voltrondata.com.invalid>
wrote:
> I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of
> integrating Substrait plans into Flight SQL if possible and it aligns
> with the arrow-adbc work.
>
> On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <li...@apache.org>
> wrote:
> > My vote: +1 (binding)
> >
> > Are any other PMC members available to take a look?
> >
> > On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> >> Fair enough. For the record, my main concern with ad-hoc conventions
> >> such as "number of milliseconds expressed as an integer" is the poor
> >> usability and the potential for confusion (not to mention that
> >> sometimes
> >> the need for a higher precision can lead to add another set of
> >> APIs, but
> >> that's unlikely to be the case here :-)).
> >>
> >> Regards
> >>
> >> Antoine.
> >>
> >>
> >> Le 07/09/2022 à 14:21, David Li a écrit :
> >>> Absent further comments on this I would rather avoid adding a
> >>> potentially breaking (even if likely compatible) change to the
> >>> schema of this endpoint, if that's acceptable. I don't think a
> >>> millisecond timeout is all too different from floating-point
> >>> seconds (especially at the scale of network RPCs).
> >>>
> >>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> >>>> We could add a new type code to the union. Presumably consumers
> >>>> would
> >>>> just error on or ignore such values (the libraries just hand the
> >>>> Arrow
> >>>> array to the application, so it's up to the application what to
> >>>> do with
> >>>> an unknown type code). (And for a new consumer talking to an old
> >>>> server, the new type code would just never come up, so the only
> >>>> issue
> >>>> would be if it strictly validates the returned schema.)
> >>>>
> >>>> If there's support, I can make this revision as well.
> >>>>
> >>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> >>>>> Le 06/09/2022 à 17:21, David Li a écrit :
> >>>>>> Thanks Antoine!
> >>>>>>
> >>>>>> I've updated the PR (except for the comment about timeout
> >>>>>> units, since SqlInfo values can't be doubles/floats unless we
> >>>>>> change the schema there)
> >>>>>
> >>>>> Can we change the schema in a backwards-compatible way?
>
>
Re: [VOTE] Substrait for Flight SQL
Posted by Matthew Topol <ma...@voltrondata.com.INVALID>.
I'm not PMC but i'll give a +1 (non-binding) vote. I like the idea of
integrating Substrait plans into Flight SQL if possible and it aligns
with the arrow-adbc work.
On Thu, Sep 8 2022 at 11:31:59 AM -0400, David Li <li...@apache.org>
wrote:
> My vote: +1 (binding)
>
> Are any other PMC members available to take a look?
>
> On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
>> Fair enough. For the record, my main concern with ad-hoc conventions
>> such as "number of milliseconds expressed as an integer" is the poor
>> usability and the potential for confusion (not to mention that
>> sometimes
>> the need for a higher precision can lead to add another set of
>> APIs, but
>> that's unlikely to be the case here :-)).
>>
>> Regards
>>
>> Antoine.
>>
>>
>> Le 07/09/2022 à 14:21, David Li a écrit :
>>> Absent further comments on this I would rather avoid adding a
>>> potentially breaking (even if likely compatible) change to the
>>> schema of this endpoint, if that's acceptable. I don't think a
>>> millisecond timeout is all too different from floating-point
>>> seconds (especially at the scale of network RPCs).
>>>
>>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>>>> We could add a new type code to the union. Presumably consumers
>>>> would
>>>> just error on or ignore such values (the libraries just hand the
>>>> Arrow
>>>> array to the application, so it's up to the application what to
>>>> do with
>>>> an unknown type code). (And for a new consumer talking to an old
>>>> server, the new type code would just never come up, so the only
>>>> issue
>>>> would be if it strictly validates the returned schema.)
>>>>
>>>> If there's support, I can make this revision as well.
>>>>
>>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>>>>> Le 06/09/2022 à 17:21, David Li a écrit :
>>>>>> Thanks Antoine!
>>>>>>
>>>>>> I've updated the PR (except for the comment about timeout
>>>>>> units, since SqlInfo values can't be doubles/floats unless we
>>>>>> change the schema there)
>>>>>
>>>>> Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
My vote: +1 (binding)
Are any other PMC members available to take a look?
On Wed, Sep 7, 2022, at 09:18, Antoine Pitrou wrote:
> Fair enough. For the record, my main concern with ad-hoc conventions
> such as "number of milliseconds expressed as an integer" is the poor
> usability and the potential for confusion (not to mention that sometimes
> the need for a higher precision can lead to add another set of APIs, but
> that's unlikely to be the case here :-)).
>
> Regards
>
> Antoine.
>
>
> Le 07/09/2022 à 14:21, David Li a écrit :
>> Absent further comments on this I would rather avoid adding a potentially breaking (even if likely compatible) change to the schema of this endpoint, if that's acceptable. I don't think a millisecond timeout is all too different from floating-point seconds (especially at the scale of network RPCs).
>>
>> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>>> We could add a new type code to the union. Presumably consumers would
>>> just error on or ignore such values (the libraries just hand the Arrow
>>> array to the application, so it's up to the application what to do with
>>> an unknown type code). (And for a new consumer talking to an old
>>> server, the new type code would just never come up, so the only issue
>>> would be if it strictly validates the returned schema.)
>>>
>>> If there's support, I can make this revision as well.
>>>
>>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>>>> Le 06/09/2022 à 17:21, David Li a écrit :
>>>>> Thanks Antoine!
>>>>>
>>>>> I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
>>>>
>>>> Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by Antoine Pitrou <an...@python.org>.
Fair enough. For the record, my main concern with ad-hoc conventions
such as "number of milliseconds expressed as an integer" is the poor
usability and the potential for confusion (not to mention that sometimes
the need for a higher precision can lead to add another set of APIs, but
that's unlikely to be the case here :-)).
Regards
Antoine.
Le 07/09/2022 à 14:21, David Li a écrit :
> Absent further comments on this I would rather avoid adding a potentially breaking (even if likely compatible) change to the schema of this endpoint, if that's acceptable. I don't think a millisecond timeout is all too different from floating-point seconds (especially at the scale of network RPCs).
>
> On Tue, Sep 6, 2022, at 12:44, David Li wrote:
>> We could add a new type code to the union. Presumably consumers would
>> just error on or ignore such values (the libraries just hand the Arrow
>> array to the application, so it's up to the application what to do with
>> an unknown type code). (And for a new consumer talking to an old
>> server, the new type code would just never come up, so the only issue
>> would be if it strictly validates the returned schema.)
>>
>> If there's support, I can make this revision as well.
>>
>> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>>> Le 06/09/2022 à 17:21, David Li a écrit :
>>>> Thanks Antoine!
>>>>
>>>> I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
>>>
>>> Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
Absent further comments on this I would rather avoid adding a potentially breaking (even if likely compatible) change to the schema of this endpoint, if that's acceptable. I don't think a millisecond timeout is all too different from floating-point seconds (especially at the scale of network RPCs).
On Tue, Sep 6, 2022, at 12:44, David Li wrote:
> We could add a new type code to the union. Presumably consumers would
> just error on or ignore such values (the libraries just hand the Arrow
> array to the application, so it's up to the application what to do with
> an unknown type code). (And for a new consumer talking to an old
> server, the new type code would just never come up, so the only issue
> would be if it strictly validates the returned schema.)
>
> If there's support, I can make this revision as well.
>
> On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
>> Le 06/09/2022 à 17:21, David Li a écrit :
>>> Thanks Antoine!
>>>
>>> I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
>>
>> Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
We could add a new type code to the union. Presumably consumers would just error on or ignore such values (the libraries just hand the Arrow array to the application, so it's up to the application what to do with an unknown type code). (And for a new consumer talking to an old server, the new type code would just never come up, so the only issue would be if it strictly validates the returned schema.)
If there's support, I can make this revision as well.
On Tue, Sep 6, 2022, at 12:37, Antoine Pitrou wrote:
> Le 06/09/2022 à 17:21, David Li a écrit :
>> Thanks Antoine!
>>
>> I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
>
> Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by Antoine Pitrou <an...@python.org>.
Le 06/09/2022 à 17:21, David Li a écrit :
> Thanks Antoine!
>
> I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
Can we change the schema in a backwards-compatible way?
Re: [VOTE] Substrait for Flight SQL
Posted by David Li <li...@apache.org>.
Thanks Antoine!
I've updated the PR (except for the comment about timeout units, since SqlInfo values can't be doubles/floats unless we change the schema there)
On Tue, Sep 6, 2022, at 09:24, Antoine Pitrou wrote:
> Hi,
>
> Sorry for the delay. I took the time to read the protobuf definitions
> again and posted a few (relatively minor) comments in the PR.
>
> On the principle the spec looks sound so I'm giving this a +1 (binding).
>
> Regards
>
> Antoine.
>
>
> Le 01/09/2022 à 01:51, David Li a écrit :
>> Hello,
>>
>> I am proposing to extend the Flight SQL specification with the following features:
>>
>> - Support for queries using Substrait [1]
>> - Explicit transaction RPCs
>> - Explicit cancellation of distributed queries
>>
>> The proposal was discussed previously in [2]. The new Protobuf definitions, an implementation for C++ and Java, and integration tests, are available at [3] with some further discussion.
>>
>> The vote will be open for at least 72 hours.
>>
>> [ ] +1 : Accept the proposal
>> [ ] 0 : No opinion
>> [ ] -1 : Do not accept this proposal because...
>>
>> [1]: https://substrait.io/
>> [2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
>> [3]: https://github.com/apache/arrow/pull/13492
Re: [VOTE] Substrait for Flight SQL
Posted by Antoine Pitrou <an...@python.org>.
Hi,
Sorry for the delay. I took the time to read the protobuf definitions
again and posted a few (relatively minor) comments in the PR.
On the principle the spec looks sound so I'm giving this a +1 (binding).
Regards
Antoine.
Le 01/09/2022 à 01:51, David Li a écrit :
> Hello,
>
> I am proposing to extend the Flight SQL specification with the following features:
>
> - Support for queries using Substrait [1]
> - Explicit transaction RPCs
> - Explicit cancellation of distributed queries
>
> The proposal was discussed previously in [2]. The new Protobuf definitions, an implementation for C++ and Java, and integration tests, are available at [3] with some further discussion.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 : Accept the proposal
> [ ] 0 : No opinion
> [ ] -1 : Do not accept this proposal because...
>
> [1]: https://substrait.io/
> [2]: https://lists.apache.org/thread/l6j0nc43y7czs2m4kpzrckmrtgr9qs9z
> [3]: https://github.com/apache/arrow/pull/13492