You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Dhruv Gohil <yo...@gmail.com> on 2015/12/29 10:24:40 UTC
Re: Python Driver Contribution Idea
Just my 2cents:
I think its into greater good that J/ODBC/Native/REST connectivity
features (for SQL operations) are pushed under "Apache Calcite" Project
instead of Drill kind of projects.
Just like "Apache Phoenix <https://phoenix.apache.org/server.html>" guys
started with Query Server
<https://issues.apache.org/jira/browse/PHOENIX-971>
Drill can/should use Calcite+Avatica
<http://calcite.apache.org/docs/avatica_overview.html> combination to
support native language interfaces.
AFAIK ODBC/Native client development is in TODO and seeking contribution
effort, so if Wojtek selects to work with Calcite Community on this, All
dependent project including DRILL will get the benefit of Python
Connectivity.
cc:dev@calcite.apache.org
On Monday 28 December 2015 01:37 PM, Wojciech Nowak wrote:
> Dear Drill developers,
>
> Recently I was trying to use Drill from Python through ODBC interface based on blog post from https://www.mapr.com/blog/using-drill-programmatically-python-r-and-perl It worked as expected, but what struck to me was that It’s a lot of hassle to configure it.
>
> That’s why based on Your site under Contribution Ideas (https://drill.apache.org/docs/apache-drill-contribution-ideas/) I decided to create simpler solution for Python community.
>
> My Contribution would have two phases:
> client/driver for interacting with Drill
> dsl which will provide a easier and idiomatic way to write and manipulate queries using defined query set expressions.
>
>
> 1.
> Similarly to official client for Elastic Search (https://github.com/elastic/elasticsearch-py) I would like to use Rest-Api of Drill for which i found documentation under https://drill.apache.org/docs/rest-api/
> sketch of usage:
> https://gist.github.com/PythonicNinja/9b4952b6cbc17572c7db#file-pydrill-py
>
> questions:
> 1.1 I was wondering if Python driver for Drill could be based on Rest-Api, do you see any problems?
> 1.2 Do you have any ideas or suggestions for that project?
>
> 2.
> It would be separate package from driver, you can install as an optional package via command:
> pip install pydrill-dsl
> so that it would have separate releases from 1 package.
> It would enhance way of interacting with Drill via query set like expressions.
> sketch of usage:
> https://gist.github.com/PythonicNinja/9b4952b6cbc17572c7db#file-pydrill_dsl-py
>
> questions:
> 2.1 Should it be separated from Python Drill Driver package?
> 2.2 Do you have any ideas or suggestions for that project?
>
> This contribution would be part of my Master Thesis, so any ideas are welcome. My thesis supervisor suggested to contact You to get Drill core developers perspective.
>
> I would be very grateful if You could provide me with your thoughts.
>
> kind regards,
> Wojtek Nowak
>
Re: Python Driver Contribution Idea
Posted by Josh Elser <jo...@gmail.com>.
Ted Dunning wrote:
> On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil<yo...@gmail.com>
> wrote:
>
>> Just my 2cents:
>> I think its into greater good that J/ODBC/Native/REST connectivity
>> features (for SQL operations) are pushed under "Apache Calcite" Project
>> instead of Drill kind of projects.
>> Just like "Apache Phoenix<https://phoenix.apache.org/server.html>" guys
>> started with Query Server<
>> https://issues.apache.org/jira/browse/PHOENIX-971>
>>
>> Drill can/should use Calcite+Avatica<
>> http://calcite.apache.org/docs/avatica_overview.html> combination to
>> support native language interfaces.
>> AFAIK ODBC/Native client development is in TODO and seeking contribution
>> effort, so if Wojtek selects to work with Calcite Community on this, All
>> dependent project including DRILL will get the benefit of Python
>> Connectivity.
>>
>
> Doing the development first in Drill and then porting it back to Calcite if
> (when) Calcite gets the rest interface going seems like a better path to
> success since it gets results sooner.
>
FWIW, I think I'll have some cycles to start looking into a Python
implementation in Avatica itself "soonish". I haven't really looked into
Drill's current relationship with Calcite, but would be amenable to
having discussions on what a Python API would look like (the ES project
Wojciech linked and Cassandra are two places I plan to look for
inspiration).
Re: Python Driver Contribution Idea
Posted by Josh Elser <jo...@gmail.com>.
Ted Dunning wrote:
> On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil<yo...@gmail.com>
> wrote:
>
>> Just my 2cents:
>> I think its into greater good that J/ODBC/Native/REST connectivity
>> features (for SQL operations) are pushed under "Apache Calcite" Project
>> instead of Drill kind of projects.
>> Just like "Apache Phoenix<https://phoenix.apache.org/server.html>" guys
>> started with Query Server<
>> https://issues.apache.org/jira/browse/PHOENIX-971>
>>
>> Drill can/should use Calcite+Avatica<
>> http://calcite.apache.org/docs/avatica_overview.html> combination to
>> support native language interfaces.
>> AFAIK ODBC/Native client development is in TODO and seeking contribution
>> effort, so if Wojtek selects to work with Calcite Community on this, All
>> dependent project including DRILL will get the benefit of Python
>> Connectivity.
>>
>
> Doing the development first in Drill and then porting it back to Calcite if
> (when) Calcite gets the rest interface going seems like a better path to
> success since it gets results sooner.
>
FWIW, I think I'll have some cycles to start looking into a Python
implementation in Avatica itself "soonish". I haven't really looked into
Drill's current relationship with Calcite, but would be amenable to
having discussions on what a Python API would look like (the ES project
Wojciech linked and Cassandra are two places I plan to look for
inspiration).
Re: Python Driver Contribution Idea
Posted by Josh Elser <jo...@gmail.com>.
Ted Dunning wrote:
> On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil<yo...@gmail.com>
> wrote:
>
>> Just my 2cents:
>> I think its into greater good that J/ODBC/Native/REST connectivity
>> features (for SQL operations) are pushed under "Apache Calcite" Project
>> instead of Drill kind of projects.
>> Just like "Apache Phoenix<https://phoenix.apache.org/server.html>" guys
>> started with Query Server<
>> https://issues.apache.org/jira/browse/PHOENIX-971>
>>
>> Drill can/should use Calcite+Avatica<
>> http://calcite.apache.org/docs/avatica_overview.html> combination to
>> support native language interfaces.
>> AFAIK ODBC/Native client development is in TODO and seeking contribution
>> effort, so if Wojtek selects to work with Calcite Community on this, All
>> dependent project including DRILL will get the benefit of Python
>> Connectivity.
>>
>
> Doing the development first in Drill and then porting it back to Calcite if
> (when) Calcite gets the rest interface going seems like a better path to
> success since it gets results sooner.
>
FWIW, I think I'll have some cycles to start looking into a Python
implementation in Avatica itself "soonish". I haven't really looked into
Drill's current relationship with Calcite, but would be amenable to
having discussions on what a Python API would look like (the ES project
Wojciech linked and Cassandra are two places I plan to look for
inspiration).
Re: Python Driver Contribution Idea
Posted by Dhruv Gohil <yo...@gmail.com>.
+1 on "gets results sooner" path!
On Tuesday 29 December 2015 08:07 PM, Ted Dunning wrote:
> On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil <yo...@gmail.com>
> wrote:
>
>> Just my 2cents:
>> I think its into greater good that J/ODBC/Native/REST connectivity
>> features (for SQL operations) are pushed under "Apache Calcite" Project
>> instead of Drill kind of projects.
>> Just like "Apache Phoenix <https://phoenix.apache.org/server.html>" guys
>> started with Query Server <
>> https://issues.apache.org/jira/browse/PHOENIX-971>
>>
>> Drill can/should use Calcite+Avatica <
>> http://calcite.apache.org/docs/avatica_overview.html> combination to
>> support native language interfaces.
>> AFAIK ODBC/Native client development is in TODO and seeking contribution
>> effort, so if Wojtek selects to work with Calcite Community on this, All
>> dependent project including DRILL will get the benefit of Python
>> Connectivity.
>>
> Doing the development first in Drill and then porting it back to Calcite if
> (when) Calcite gets the rest interface going seems like a better path to
> success since it gets results sooner.
>
Re: Python Driver Contribution Idea
Posted by Ted Dunning <te...@gmail.com>.
On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil <yo...@gmail.com>
wrote:
> Just my 2cents:
> I think its into greater good that J/ODBC/Native/REST connectivity
> features (for SQL operations) are pushed under "Apache Calcite" Project
> instead of Drill kind of projects.
> Just like "Apache Phoenix <https://phoenix.apache.org/server.html>" guys
> started with Query Server <
> https://issues.apache.org/jira/browse/PHOENIX-971>
>
> Drill can/should use Calcite+Avatica <
> http://calcite.apache.org/docs/avatica_overview.html> combination to
> support native language interfaces.
> AFAIK ODBC/Native client development is in TODO and seeking contribution
> effort, so if Wojtek selects to work with Calcite Community on this, All
> dependent project including DRILL will get the benefit of Python
> Connectivity.
>
Doing the development first in Drill and then porting it back to Calcite if
(when) Calcite gets the rest interface going seems like a better path to
success since it gets results sooner.
Re: Python Driver Contribution Idea
Posted by Ted Dunning <te...@gmail.com>.
On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil <yo...@gmail.com>
wrote:
> Just my 2cents:
> I think its into greater good that J/ODBC/Native/REST connectivity
> features (for SQL operations) are pushed under "Apache Calcite" Project
> instead of Drill kind of projects.
> Just like "Apache Phoenix <https://phoenix.apache.org/server.html>" guys
> started with Query Server <
> https://issues.apache.org/jira/browse/PHOENIX-971>
>
> Drill can/should use Calcite+Avatica <
> http://calcite.apache.org/docs/avatica_overview.html> combination to
> support native language interfaces.
> AFAIK ODBC/Native client development is in TODO and seeking contribution
> effort, so if Wojtek selects to work with Calcite Community on this, All
> dependent project including DRILL will get the benefit of Python
> Connectivity.
>
Doing the development first in Drill and then porting it back to Calcite if
(when) Calcite gets the rest interface going seems like a better path to
success since it gets results sooner.
Re: Python Driver Contribution Idea
Posted by Ted Dunning <te...@gmail.com>.
On Tue, Dec 29, 2015 at 1:24 AM, Dhruv Gohil <yo...@gmail.com>
wrote:
> Just my 2cents:
> I think its into greater good that J/ODBC/Native/REST connectivity
> features (for SQL operations) are pushed under "Apache Calcite" Project
> instead of Drill kind of projects.
> Just like "Apache Phoenix <https://phoenix.apache.org/server.html>" guys
> started with Query Server <
> https://issues.apache.org/jira/browse/PHOENIX-971>
>
> Drill can/should use Calcite+Avatica <
> http://calcite.apache.org/docs/avatica_overview.html> combination to
> support native language interfaces.
> AFAIK ODBC/Native client development is in TODO and seeking contribution
> effort, so if Wojtek selects to work with Calcite Community on this, All
> dependent project including DRILL will get the benefit of Python
> Connectivity.
>
Doing the development first in Drill and then porting it back to Calcite if
(when) Calcite gets the rest interface going seems like a better path to
success since it gets results sooner.