You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Tanveer Ahmad - EWI <T....@tudelft.nl> on 2021/05/15 12:15:26 UTC
Python Flight example with query command
Hi all,
For Python Flight example<https://github.com/apache/arrow/tree/master/python/examples/flight>, I can start server (python server.py -> Serving on grpc+tcp://localhost:5005) and client can put (python client.py put localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005 -p mycsv.csv) command retrieves data with -p (path) option.
I am wondering how to query (like python client.py get localhost:5005 -c "select * from ? limit 10") using -c, command this data , which I had already put on server through put command.
Thanks.
Regards,
Tanveer Ahmad
Re: Python Flight example with query command
Posted by Wes McKinney <we...@gmail.com>.
It does not implement it. We don’t have FlightSQL implemented in Python yet
either but I expect this will happen within the next 12 months.
Contributions for this always welcome
On Tue, May 18, 2021 at 12:51 AM Tanveer Ahmad - EWI <T....@tudelft.nl>
wrote:
> Hi David,
>
>
> Thank you for the reply.
>
>
> I have found that Arrow Datafusion<
> https://github.com/apache/arrow-datafusion/tree/3be087a78846beffdbc4a9f80c73938fa18d24a7/datafusion-examples/examples#distributed>
> project offers something similar for what I am looking for. Do you think
> this project implements FlightSQL proposal?
>
>
> Regards,
> Tanveer Ahmad
> ________________________________
> From: David Li <li...@apache.org>
> Sent: Saturday, May 15, 2021 3:10:53 PM
> To: dev@arrow.apache.org
> Subject: Re: Python Flight example with query command
>
> Hey Tanveer,
>
> Something like this should work:
>
> $ python examples/flight/client.py put localhost:1234 foo.csv
> File Name: foo.csv
> Table rows= 1
> a b
> 0 1 2
> $ python examples/flight/client.py get localhost:1234 -p foo.csv
> Ticket: <Ticket b"(1, None, (b'foo.csv',))">
> <Location b'grpc+tcp://localhost:1234'>
> a b
> 0 1 2
>
> Note that Flight itself does not implement SQL query functionality or
> anything of the sort. It is a common misconception, I think
> exacerbated since Flight is often discussed in the context of products
> like Dremio which implement such functionality on top of Flight. But
> really, Flight itself is just a 'dumb pipe' for Arrow data for
> building such systems.
>
> You may be interested in the FlightSQL proposal which defines at least
> an interface for database systems to make themselves available over
> Flight and for clients to generically query them. However that
> proposal has been stalled for a while.
>
> Best,
> David
>
> On 2021/05/15 12:15:26, Tanveer Ahmad - EWI <T....@tudelft.nl> wrote:
> > Hi all,
> >
> >
> > For Python Flight example<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_arrow_tree_master_python_examples_flight&d=DwIBaQ&c=XYzUhXBD2cD-CornpT4QE19xOJBbRy-TBPLK0X9U2o8&r=0FbbJetCCSYzJEnEDCQ1rNv76vTL6SUFCukKhvNosPs&m=-rzSkXp2WuWkj32SFKyE8CQ6Z6ViEXHg1AnYkaBmj4E&s=Q0ypcfj8NSFdgVXa_c8NxxQ9dpVkQ1EQXE157BX3R7g&e=
> >, I can start server (python server.py -> Serving on
> grpc+tcp://localhost:5005) and client can put (python client.py put
> localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005
> -p mycsv.csv) command retrieves data with -p (path) option.
> >
> >
> > I am wondering how to query (like python client.py get localhost:5005 -c
> "select * from ? limit 10") using -c, command this data , which I had
> already put on server through put command.
> >
> >
> > Thanks.
> >
> > Regards,
> > Tanveer Ahmad
> >
> >
>
Re: Python Flight example with query command
Posted by Tanveer Ahmad - EWI <T....@tudelft.nl>.
Hi David,
Thank you for the reply.
I have found that Arrow Datafusion<https://github.com/apache/arrow-datafusion/tree/3be087a78846beffdbc4a9f80c73938fa18d24a7/datafusion-examples/examples#distributed> project offers something similar for what I am looking for. Do you think this project implements FlightSQL proposal?
Regards,
Tanveer Ahmad
________________________________
From: David Li <li...@apache.org>
Sent: Saturday, May 15, 2021 3:10:53 PM
To: dev@arrow.apache.org
Subject: Re: Python Flight example with query command
Hey Tanveer,
Something like this should work:
$ python examples/flight/client.py put localhost:1234 foo.csv
File Name: foo.csv
Table rows= 1
a b
0 1 2
$ python examples/flight/client.py get localhost:1234 -p foo.csv
Ticket: <Ticket b"(1, None, (b'foo.csv',))">
<Location b'grpc+tcp://localhost:1234'>
a b
0 1 2
Note that Flight itself does not implement SQL query functionality or
anything of the sort. It is a common misconception, I think
exacerbated since Flight is often discussed in the context of products
like Dremio which implement such functionality on top of Flight. But
really, Flight itself is just a 'dumb pipe' for Arrow data for
building such systems.
You may be interested in the FlightSQL proposal which defines at least
an interface for database systems to make themselves available over
Flight and for clients to generically query them. However that
proposal has been stalled for a while.
Best,
David
On 2021/05/15 12:15:26, Tanveer Ahmad - EWI <T....@tudelft.nl> wrote:
> Hi all,
>
>
> For Python Flight example<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_arrow_tree_master_python_examples_flight&d=DwIBaQ&c=XYzUhXBD2cD-CornpT4QE19xOJBbRy-TBPLK0X9U2o8&r=0FbbJetCCSYzJEnEDCQ1rNv76vTL6SUFCukKhvNosPs&m=-rzSkXp2WuWkj32SFKyE8CQ6Z6ViEXHg1AnYkaBmj4E&s=Q0ypcfj8NSFdgVXa_c8NxxQ9dpVkQ1EQXE157BX3R7g&e= >, I can start server (python server.py -> Serving on grpc+tcp://localhost:5005) and client can put (python client.py put localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005 -p mycsv.csv) command retrieves data with -p (path) option.
>
>
> I am wondering how to query (like python client.py get localhost:5005 -c "select * from ? limit 10") using -c, command this data , which I had already put on server through put command.
>
>
> Thanks.
>
> Regards,
> Tanveer Ahmad
>
>
Re: Python Flight example with query command
Posted by David Li <li...@apache.org>.
Hey Tanveer,
Something like this should work:
$ python examples/flight/client.py put localhost:1234 foo.csv
File Name: foo.csv
Table rows= 1
a b
0 1 2
$ python examples/flight/client.py get localhost:1234 -p foo.csv
Ticket: <Ticket b"(1, None, (b'foo.csv',))">
<Location b'grpc+tcp://localhost:1234'>
a b
0 1 2
Note that Flight itself does not implement SQL query functionality or
anything of the sort. It is a common misconception, I think
exacerbated since Flight is often discussed in the context of products
like Dremio which implement such functionality on top of Flight. But
really, Flight itself is just a 'dumb pipe' for Arrow data for
building such systems.
You may be interested in the FlightSQL proposal which defines at least
an interface for database systems to make themselves available over
Flight and for clients to generically query them. However that
proposal has been stalled for a while.
Best,
David
On 2021/05/15 12:15:26, Tanveer Ahmad - EWI <T....@tudelft.nl> wrote:
> Hi all,
>
>
> For Python Flight example<https://github.com/apache/arrow/tree/master/python/examples/flight>, I can start server (python server.py -> Serving on grpc+tcp://localhost:5005) and client can put (python client.py put localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005 -p mycsv.csv) command retrieves data with -p (path) option.
>
>
> I am wondering how to query (like python client.py get localhost:5005 -c "select * from ? limit 10") using -c, command this data , which I had already put on server through put command.
>
>
> Thanks.
>
> Regards,
> Tanveer Ahmad
>
>