You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by David Li <li...@gmail.com> on 2019/07/08 14:33:37 UTC
[Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Hi all,
I've put together two more proposals for Flight, motivated by projects
we've been working on. I'd appreciate any comments on the
design/reasoning; I'm already working on the implementation, alongside
some other improvements to Flight.
The first is to modify the DoPut call to follow the same request
pattern as DoGet. This is a format change and would require a vote.
https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
---
The second is to introduce a middleware API for Flight, to make
integration with logging and tracing frameworks (e.g. OpenTracing)
easier:
https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
As part of this, I'm working on APIs to send richer error
codes/messages between Flight clients/servers, now that @Micah has
enabled custom error codes in Arrow-C++. These are API changes and
should not require a vote, I believe.
---
Anyone should have comment access to these documents, but let me know if not.
Thanks!
Best,
David
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Wes McKinney <we...@gmail.com>.
We could start a contrib/ folder in the root of the project, for lack
of an obvious place to put code that isn't part of any core library
On Mon, Jul 15, 2019 at 9:19 AM David Li <li...@gmail.com> wrote:
>
> Hi all,
>
> Thanks for the comments so far. Does anyone else have thoughts on
> either proposal? For middleware, I can put up an implementation draft
> if that's more useful. I could also include an example OpenTracing
> integration, though I'm not sure how to structure "contrib" modules
> (presumably we don't want that as an actual dependency).
>
> Thanks,
> David
>
> On 7/8/19, Wes McKinney <we...@gmail.com> wrote:
> > hi David,
> >
> > On the DoPut change, I am potentially in favor of making a change to
> > support the use case you are describing. In principle, the CMD field
> > of FlightDescriptor could be used to transmit application-defined
> > information with a Put request. Though I suspect that a DoAction to
> > "prepare" the server for a DoPut would be more common in production
> > servers. I'm interested in Jacques's or others' thoughts about this
> >
> > On the middleware API I am in favor; there are obviously a lot details
> > that have to be worked out, but it's important that Flight servers and
> > clients are able to be instrumented to collect metrics in a production
> > setting.
> >
> > Thanks,
> > Wes
> >
> > On Mon, Jul 8, 2019 at 9:33 AM David Li <li...@gmail.com> wrote:
> >>
> >> Hi all,
> >>
> >> I've put together two more proposals for Flight, motivated by projects
> >> we've been working on. I'd appreciate any comments on the
> >> design/reasoning; I'm already working on the implementation, alongside
> >> some other improvements to Flight.
> >>
> >> The first is to modify the DoPut call to follow the same request
> >> pattern as DoGet. This is a format change and would require a vote.
> >>
> >> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
> >>
> >> ---
> >>
> >> The second is to introduce a middleware API for Flight, to make
> >> integration with logging and tracing frameworks (e.g. OpenTracing)
> >> easier:
> >>
> >> https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
> >>
> >> As part of this, I'm working on APIs to send richer error
> >> codes/messages between Flight clients/servers, now that @Micah has
> >> enabled custom error codes in Arrow-C++. These are API changes and
> >> should not require a vote, I believe.
> >>
> >> ---
> >>
> >> Anyone should have comment access to these documents, but let me know if
> >> not.
> >>
> >> Thanks!
> >>
> >> Best,
> >> David
> >
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by David Li <li...@gmail.com>.
Hi all,
Thanks for the comments so far. Does anyone else have thoughts on
either proposal? For middleware, I can put up an implementation draft
if that's more useful. I could also include an example OpenTracing
integration, though I'm not sure how to structure "contrib" modules
(presumably we don't want that as an actual dependency).
Thanks,
David
On 7/8/19, Wes McKinney <we...@gmail.com> wrote:
> hi David,
>
> On the DoPut change, I am potentially in favor of making a change to
> support the use case you are describing. In principle, the CMD field
> of FlightDescriptor could be used to transmit application-defined
> information with a Put request. Though I suspect that a DoAction to
> "prepare" the server for a DoPut would be more common in production
> servers. I'm interested in Jacques's or others' thoughts about this
>
> On the middleware API I am in favor; there are obviously a lot details
> that have to be worked out, but it's important that Flight servers and
> clients are able to be instrumented to collect metrics in a production
> setting.
>
> Thanks,
> Wes
>
> On Mon, Jul 8, 2019 at 9:33 AM David Li <li...@gmail.com> wrote:
>>
>> Hi all,
>>
>> I've put together two more proposals for Flight, motivated by projects
>> we've been working on. I'd appreciate any comments on the
>> design/reasoning; I'm already working on the implementation, alongside
>> some other improvements to Flight.
>>
>> The first is to modify the DoPut call to follow the same request
>> pattern as DoGet. This is a format change and would require a vote.
>>
>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>>
>> ---
>>
>> The second is to introduce a middleware API for Flight, to make
>> integration with logging and tracing frameworks (e.g. OpenTracing)
>> easier:
>>
>> https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
>>
>> As part of this, I'm working on APIs to send richer error
>> codes/messages between Flight clients/servers, now that @Micah has
>> enabled custom error codes in Arrow-C++. These are API changes and
>> should not require a vote, I believe.
>>
>> ---
>>
>> Anyone should have comment access to these documents, but let me know if
>> not.
>>
>> Thanks!
>>
>> Best,
>> David
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Wes McKinney <we...@gmail.com>.
You can e-mail dev-unsubscribe@arrow.apache.org. You might consider
setting up an e-mail filter so you have the option to participate in
community discussions occasionally
On Mon, Jul 8, 2019 at 1:05 PM Jonathan Spooner <js...@gmail.com> wrote:
>
> How do you unsubscribe from these emails?
>
> On Mon, Jul 8, 2019 at 10:00 AM Wes McKinney <we...@gmail.com> wrote:
>
> > hi David,
> >
> > On the DoPut change, I am potentially in favor of making a change to
> > support the use case you are describing. In principle, the CMD field
> > of FlightDescriptor could be used to transmit application-defined
> > information with a Put request. Though I suspect that a DoAction to
> > "prepare" the server for a DoPut would be more common in production
> > servers. I'm interested in Jacques's or others' thoughts about this
> >
> > On the middleware API I am in favor; there are obviously a lot details
> > that have to be worked out, but it's important that Flight servers and
> > clients are able to be instrumented to collect metrics in a production
> > setting.
> >
> > Thanks,
> > Wes
> >
> > On Mon, Jul 8, 2019 at 9:33 AM David Li <li...@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > I've put together two more proposals for Flight, motivated by projects
> > > we've been working on. I'd appreciate any comments on the
> > > design/reasoning; I'm already working on the implementation, alongside
> > > some other improvements to Flight.
> > >
> > > The first is to modify the DoPut call to follow the same request
> > > pattern as DoGet. This is a format change and would require a vote.
> > >
> > >
> > https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
> > >
> > > ---
> > >
> > > The second is to introduce a middleware API for Flight, to make
> > > integration with logging and tracing frameworks (e.g. OpenTracing)
> > > easier:
> > >
> > >
> > https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
> > >
> > > As part of this, I'm working on APIs to send richer error
> > > codes/messages between Flight clients/servers, now that @Micah has
> > > enabled custom error codes in Arrow-C++. These are API changes and
> > > should not require a vote, I believe.
> > >
> > > ---
> > >
> > > Anyone should have comment access to these documents, but let me know if
> > not.
> > >
> > > Thanks!
> > >
> > > Best,
> > > David
> >
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Jonathan Spooner <js...@gmail.com>.
How do you unsubscribe from these emails?
On Mon, Jul 8, 2019 at 10:00 AM Wes McKinney <we...@gmail.com> wrote:
> hi David,
>
> On the DoPut change, I am potentially in favor of making a change to
> support the use case you are describing. In principle, the CMD field
> of FlightDescriptor could be used to transmit application-defined
> information with a Put request. Though I suspect that a DoAction to
> "prepare" the server for a DoPut would be more common in production
> servers. I'm interested in Jacques's or others' thoughts about this
>
> On the middleware API I am in favor; there are obviously a lot details
> that have to be worked out, but it's important that Flight servers and
> clients are able to be instrumented to collect metrics in a production
> setting.
>
> Thanks,
> Wes
>
> On Mon, Jul 8, 2019 at 9:33 AM David Li <li...@gmail.com> wrote:
> >
> > Hi all,
> >
> > I've put together two more proposals for Flight, motivated by projects
> > we've been working on. I'd appreciate any comments on the
> > design/reasoning; I'm already working on the implementation, alongside
> > some other improvements to Flight.
> >
> > The first is to modify the DoPut call to follow the same request
> > pattern as DoGet. This is a format change and would require a vote.
> >
> >
> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
> >
> > ---
> >
> > The second is to introduce a middleware API for Flight, to make
> > integration with logging and tracing frameworks (e.g. OpenTracing)
> > easier:
> >
> >
> https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
> >
> > As part of this, I'm working on APIs to send richer error
> > codes/messages between Flight clients/servers, now that @Micah has
> > enabled custom error codes in Arrow-C++. These are API changes and
> > should not require a vote, I believe.
> >
> > ---
> >
> > Anyone should have comment access to these documents, but let me know if
> not.
> >
> > Thanks!
> >
> > Best,
> > David
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Wes McKinney <we...@gmail.com>.
hi David,
On the DoPut change, I am potentially in favor of making a change to
support the use case you are describing. In principle, the CMD field
of FlightDescriptor could be used to transmit application-defined
information with a Put request. Though I suspect that a DoAction to
"prepare" the server for a DoPut would be more common in production
servers. I'm interested in Jacques's or others' thoughts about this
On the middleware API I am in favor; there are obviously a lot details
that have to be worked out, but it's important that Flight servers and
clients are able to be instrumented to collect metrics in a production
setting.
Thanks,
Wes
On Mon, Jul 8, 2019 at 9:33 AM David Li <li...@gmail.com> wrote:
>
> Hi all,
>
> I've put together two more proposals for Flight, motivated by projects
> we've been working on. I'd appreciate any comments on the
> design/reasoning; I'm already working on the implementation, alongside
> some other improvements to Flight.
>
> The first is to modify the DoPut call to follow the same request
> pattern as DoGet. This is a format change and would require a vote.
>
> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>
> ---
>
> The second is to introduce a middleware API for Flight, to make
> integration with logging and tracing frameworks (e.g. OpenTracing)
> easier:
>
> https://docs.google.com/document/d/1Nm1ASp6w5PZQX42KkOTLK5LvrOv5q2cCbkaiVjqKjLU/edit?usp=sharing
>
> As part of this, I'm working on APIs to send richer error
> codes/messages between Flight clients/servers, now that @Micah has
> enabled custom error codes in Arrow-C++. These are API changes and
> should not require a vote, I believe.
>
> ---
>
> Anyone should have comment access to these documents, but let me know if not.
>
> Thanks!
>
> Best,
> David
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by David Li <li...@gmail.com>.
Following up on this, I've put up the C++/Python implementation as
well: https://github.com/apache/arrow/pull/5552
The two are likely not fully compatible, since I'm aware of at least
one case where the gRPC implementations differ - I want to set up real
integration tests for these things as part of ARROW-5875
(https://issues.apache.org/jira/browse/ARROW-5875).
For the DoPut discussion, I'm going to table that for now unless
there's more interest, as I'd like to focus on this (and another
Flight feature that I've been working on a proposal for).
Best,
David
On 9/9/19, David Li <li...@gmail.com> wrote:
> I'm happy to start a new thread to focus on DoPut specifically.
> Middleware for Java has been in review.
>
> Best,
> David
>
> On 9/9/19, Wes McKinney <we...@gmail.com> wrote:
>> Ah, I think I'm referring to the format change around DoPut, for which
>> there is not a PR yet. Sorry for my confusion
>>
>> Do we want to start a separate discussion thread about that?
>>
>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>>
>> On Mon, Sep 9, 2019 at 3:51 PM Antoine Pitrou <an...@python.org> wrote:
>>>
>>>
>>> Isn't a middleware an implementation-specific concern? Does it need a
>>> formal vote?
>>>
>>> Regards
>>>
>>> Antoine.
>>>
>>>
>>> Le 09/09/2019 à 22:49, Wes McKinney a écrit :
>>> > It seems like there is positive feedback on the PR. Do we want to have
>>> > a vote about this?
>>> >
>>> > On Mon, Aug 12, 2019 at 7:54 AM David Li <li...@gmail.com>
>>> > wrote:
>>> >>
>>> >> I've (finally) put up a draft implementation of middleware for Java:
>>> >> https://github.com/apache/arrow/pull/5068
>>> >>
>>> >> Hopefully this helps clarify how the proposal works.
>>> >>
>>> >> Best,
>>> >> David
>>> >>
>>> >> On 7/25/19, David Li <li...@gmail.com> wrote:
>>> >>> Thanks for the feedback, Antoine. That would be a natural method to
>>> >>> have - then the server could deny uploads (as you mention) or note
>>> >>> that the stream already exists. I've updated the proposal to reflect
>>> >>> that, leaving more detailed semantics (e.g. append vs overwrite)
>>> >>> application-defined.
>>> >>>
>>> >>> Best,
>>> >>> David
>>> >>>
>>> >>> On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
>>> >>>>
>>> >>>> Le 08/07/2019 à 16:33, David Li a écrit :
>>> >>>>> Hi all,
>>> >>>>>
>>> >>>>> I've put together two more proposals for Flight, motivated by
>>> >>>>> projects
>>> >>>>> we've been working on. I'd appreciate any comments on the
>>> >>>>> design/reasoning; I'm already working on the implementation,
>>> >>>>> alongside
>>> >>>>> some other improvements to Flight.
>>> >>>>>
>>> >>>>> The first is to modify the DoPut call to follow the same request
>>> >>>>> pattern as DoGet. This is a format change and would require a
>>> >>>>> vote.
>>> >>>>>
>>> >>>>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>>> >>>>
>>> >>>> It seems it would be useful to introduce a GetPutInfo (or
>>> >>>> GetUploadInfo)
>>> >>>> so as to allow differential behaviour between getting and putting.
>>> >>>>
>>> >>>> (one trivial case would be to disallow uploading altogether :-)))
>>> >>>>
>>> >>>> Regards
>>> >>>>
>>> >>>> Antoine.
>>> >>>>
>>> >>>
>>
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by David Li <li...@gmail.com>.
I'm happy to start a new thread to focus on DoPut specifically.
Middleware for Java has been in review.
Best,
David
On 9/9/19, Wes McKinney <we...@gmail.com> wrote:
> Ah, I think I'm referring to the format change around DoPut, for which
> there is not a PR yet. Sorry for my confusion
>
> Do we want to start a separate discussion thread about that?
>
> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>
> On Mon, Sep 9, 2019 at 3:51 PM Antoine Pitrou <an...@python.org> wrote:
>>
>>
>> Isn't a middleware an implementation-specific concern? Does it need a
>> formal vote?
>>
>> Regards
>>
>> Antoine.
>>
>>
>> Le 09/09/2019 à 22:49, Wes McKinney a écrit :
>> > It seems like there is positive feedback on the PR. Do we want to have
>> > a vote about this?
>> >
>> > On Mon, Aug 12, 2019 at 7:54 AM David Li <li...@gmail.com> wrote:
>> >>
>> >> I've (finally) put up a draft implementation of middleware for Java:
>> >> https://github.com/apache/arrow/pull/5068
>> >>
>> >> Hopefully this helps clarify how the proposal works.
>> >>
>> >> Best,
>> >> David
>> >>
>> >> On 7/25/19, David Li <li...@gmail.com> wrote:
>> >>> Thanks for the feedback, Antoine. That would be a natural method to
>> >>> have - then the server could deny uploads (as you mention) or note
>> >>> that the stream already exists. I've updated the proposal to reflect
>> >>> that, leaving more detailed semantics (e.g. append vs overwrite)
>> >>> application-defined.
>> >>>
>> >>> Best,
>> >>> David
>> >>>
>> >>> On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
>> >>>>
>> >>>> Le 08/07/2019 à 16:33, David Li a écrit :
>> >>>>> Hi all,
>> >>>>>
>> >>>>> I've put together two more proposals for Flight, motivated by
>> >>>>> projects
>> >>>>> we've been working on. I'd appreciate any comments on the
>> >>>>> design/reasoning; I'm already working on the implementation,
>> >>>>> alongside
>> >>>>> some other improvements to Flight.
>> >>>>>
>> >>>>> The first is to modify the DoPut call to follow the same request
>> >>>>> pattern as DoGet. This is a format change and would require a vote.
>> >>>>>
>> >>>>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>> >>>>
>> >>>> It seems it would be useful to introduce a GetPutInfo (or
>> >>>> GetUploadInfo)
>> >>>> so as to allow differential behaviour between getting and putting.
>> >>>>
>> >>>> (one trivial case would be to disallow uploading altogether :-)))
>> >>>>
>> >>>> Regards
>> >>>>
>> >>>> Antoine.
>> >>>>
>> >>>
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Wes McKinney <we...@gmail.com>.
Ah, I think I'm referring to the format change around DoPut, for which
there is not a PR yet. Sorry for my confusion
Do we want to start a separate discussion thread about that?
https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
On Mon, Sep 9, 2019 at 3:51 PM Antoine Pitrou <an...@python.org> wrote:
>
>
> Isn't a middleware an implementation-specific concern? Does it need a
> formal vote?
>
> Regards
>
> Antoine.
>
>
> Le 09/09/2019 à 22:49, Wes McKinney a écrit :
> > It seems like there is positive feedback on the PR. Do we want to have
> > a vote about this?
> >
> > On Mon, Aug 12, 2019 at 7:54 AM David Li <li...@gmail.com> wrote:
> >>
> >> I've (finally) put up a draft implementation of middleware for Java:
> >> https://github.com/apache/arrow/pull/5068
> >>
> >> Hopefully this helps clarify how the proposal works.
> >>
> >> Best,
> >> David
> >>
> >> On 7/25/19, David Li <li...@gmail.com> wrote:
> >>> Thanks for the feedback, Antoine. That would be a natural method to
> >>> have - then the server could deny uploads (as you mention) or note
> >>> that the stream already exists. I've updated the proposal to reflect
> >>> that, leaving more detailed semantics (e.g. append vs overwrite)
> >>> application-defined.
> >>>
> >>> Best,
> >>> David
> >>>
> >>> On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
> >>>>
> >>>> Le 08/07/2019 à 16:33, David Li a écrit :
> >>>>> Hi all,
> >>>>>
> >>>>> I've put together two more proposals for Flight, motivated by projects
> >>>>> we've been working on. I'd appreciate any comments on the
> >>>>> design/reasoning; I'm already working on the implementation, alongside
> >>>>> some other improvements to Flight.
> >>>>>
> >>>>> The first is to modify the DoPut call to follow the same request
> >>>>> pattern as DoGet. This is a format change and would require a vote.
> >>>>>
> >>>>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
> >>>>
> >>>> It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
> >>>> so as to allow differential behaviour between getting and putting.
> >>>>
> >>>> (one trivial case would be to disallow uploading altogether :-)))
> >>>>
> >>>> Regards
> >>>>
> >>>> Antoine.
> >>>>
> >>>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut
tickets
Posted by Antoine Pitrou <an...@python.org>.
Isn't a middleware an implementation-specific concern? Does it need a
formal vote?
Regards
Antoine.
Le 09/09/2019 à 22:49, Wes McKinney a écrit :
> It seems like there is positive feedback on the PR. Do we want to have
> a vote about this?
>
> On Mon, Aug 12, 2019 at 7:54 AM David Li <li...@gmail.com> wrote:
>>
>> I've (finally) put up a draft implementation of middleware for Java:
>> https://github.com/apache/arrow/pull/5068
>>
>> Hopefully this helps clarify how the proposal works.
>>
>> Best,
>> David
>>
>> On 7/25/19, David Li <li...@gmail.com> wrote:
>>> Thanks for the feedback, Antoine. That would be a natural method to
>>> have - then the server could deny uploads (as you mention) or note
>>> that the stream already exists. I've updated the proposal to reflect
>>> that, leaving more detailed semantics (e.g. append vs overwrite)
>>> application-defined.
>>>
>>> Best,
>>> David
>>>
>>> On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
>>>>
>>>> Le 08/07/2019 à 16:33, David Li a écrit :
>>>>> Hi all,
>>>>>
>>>>> I've put together two more proposals for Flight, motivated by projects
>>>>> we've been working on. I'd appreciate any comments on the
>>>>> design/reasoning; I'm already working on the implementation, alongside
>>>>> some other improvements to Flight.
>>>>>
>>>>> The first is to modify the DoPut call to follow the same request
>>>>> pattern as DoGet. This is a format change and would require a vote.
>>>>>
>>>>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>>>>
>>>> It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
>>>> so as to allow differential behaviour between getting and putting.
>>>>
>>>> (one trivial case would be to disallow uploading altogether :-)))
>>>>
>>>> Regards
>>>>
>>>> Antoine.
>>>>
>>>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by Wes McKinney <we...@gmail.com>.
It seems like there is positive feedback on the PR. Do we want to have
a vote about this?
On Mon, Aug 12, 2019 at 7:54 AM David Li <li...@gmail.com> wrote:
>
> I've (finally) put up a draft implementation of middleware for Java:
> https://github.com/apache/arrow/pull/5068
>
> Hopefully this helps clarify how the proposal works.
>
> Best,
> David
>
> On 7/25/19, David Li <li...@gmail.com> wrote:
> > Thanks for the feedback, Antoine. That would be a natural method to
> > have - then the server could deny uploads (as you mention) or note
> > that the stream already exists. I've updated the proposal to reflect
> > that, leaving more detailed semantics (e.g. append vs overwrite)
> > application-defined.
> >
> > Best,
> > David
> >
> > On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
> >>
> >> Le 08/07/2019 à 16:33, David Li a écrit :
> >>> Hi all,
> >>>
> >>> I've put together two more proposals for Flight, motivated by projects
> >>> we've been working on. I'd appreciate any comments on the
> >>> design/reasoning; I'm already working on the implementation, alongside
> >>> some other improvements to Flight.
> >>>
> >>> The first is to modify the DoPut call to follow the same request
> >>> pattern as DoGet. This is a format change and would require a vote.
> >>>
> >>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
> >>
> >> It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
> >> so as to allow differential behaviour between getting and putting.
> >>
> >> (one trivial case would be to disallow uploading altogether :-)))
> >>
> >> Regards
> >>
> >> Antoine.
> >>
> >
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by David Li <li...@gmail.com>.
I've (finally) put up a draft implementation of middleware for Java:
https://github.com/apache/arrow/pull/5068
Hopefully this helps clarify how the proposal works.
Best,
David
On 7/25/19, David Li <li...@gmail.com> wrote:
> Thanks for the feedback, Antoine. That would be a natural method to
> have - then the server could deny uploads (as you mention) or note
> that the stream already exists. I've updated the proposal to reflect
> that, leaving more detailed semantics (e.g. append vs overwrite)
> application-defined.
>
> Best,
> David
>
> On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
>>
>> Le 08/07/2019 à 16:33, David Li a écrit :
>>> Hi all,
>>>
>>> I've put together two more proposals for Flight, motivated by projects
>>> we've been working on. I'd appreciate any comments on the
>>> design/reasoning; I'm already working on the implementation, alongside
>>> some other improvements to Flight.
>>>
>>> The first is to modify the DoPut call to follow the same request
>>> pattern as DoGet. This is a format change and would require a vote.
>>>
>>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>>
>> It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
>> so as to allow differential behaviour between getting and putting.
>>
>> (one trivial case would be to disallow uploading altogether :-)))
>>
>> Regards
>>
>> Antoine.
>>
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut tickets
Posted by David Li <li...@gmail.com>.
Thanks for the feedback, Antoine. That would be a natural method to
have - then the server could deny uploads (as you mention) or note
that the stream already exists. I've updated the proposal to reflect
that, leaving more detailed semantics (e.g. append vs overwrite)
application-defined.
Best,
David
On 7/25/19, Antoine Pitrou <an...@python.org> wrote:
>
> Le 08/07/2019 à 16:33, David Li a écrit :
>> Hi all,
>>
>> I've put together two more proposals for Flight, motivated by projects
>> we've been working on. I'd appreciate any comments on the
>> design/reasoning; I'm already working on the implementation, alongside
>> some other improvements to Flight.
>>
>> The first is to modify the DoPut call to follow the same request
>> pattern as DoGet. This is a format change and would require a vote.
>>
>> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
>
> It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
> so as to allow differential behaviour between getting and putting.
>
> (one trivial case would be to disallow uploading altogether :-)))
>
> Regards
>
> Antoine.
>
Re: [Discuss][FlightRPC] Extensions to Flight: middleware and DoPut
tickets
Posted by Antoine Pitrou <an...@python.org>.
Le 08/07/2019 à 16:33, David Li a écrit :
> Hi all,
>
> I've put together two more proposals for Flight, motivated by projects
> we've been working on. I'd appreciate any comments on the
> design/reasoning; I'm already working on the implementation, alongside
> some other improvements to Flight.
>
> The first is to modify the DoPut call to follow the same request
> pattern as DoGet. This is a format change and would require a vote.
>
> https://docs.google.com/document/d/1hrwxNwPU1aOD_1ciRUOaGeUCyXYOmu6IxxCfY6Stj6w/edit?usp=sharing
It seems it would be useful to introduce a GetPutInfo (or GetUploadInfo)
so as to allow differential behaviour between getting and putting.
(one trivial case would be to disallow uploading altogether :-)))
Regards
Antoine.