You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Neeraja Rentachintala <nr...@maprtech.com> on 2016/05/12 15:49:13 UTC
Drill & Caravel
Hi Folks
Caravel is nice visualization tool recently open sourced by airbnb. Did
anyone try to integrate Drill and/or interested in contributing to making
this work with Drill.
https://github.com/airbnb/caravel
-Thanks
Neeraja
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
Lots of potential here.
On May 13, 2016 9:42 AM, "Neeraja Rentachintala" <
nrentachintala@maprtech.com> wrote:
> Yes, the key thing is the SQL Alchemy layer.
> I can see it more broadly being used than just Caravel.
>
> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
> wrote:
>
> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
> > enable other frameworks.
> >
> > Would one think that adaptation of SQLAlchemy to Drill is specific to
> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
> feasable
> > and usefull to map to Drill. This sounds like a broad general question. I
> > am sure there are orms from other language camps that might want Drill
> > backends.
> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
> >
> > > I will be looking into this as well, thanks for sharing!
> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
> wrote:
> > >
> > > > I Hi Neeraja,
> > > >
> > > > I am interested in contributing if integration is not available.
> > > > Kindly let me know
> > > >
> > > > Regards,
> > > > Nirav
> > > >
> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > > nrentachintala@maprtech.com> wrote:
> > > >
> > > > > Hi Folks
> > > > >
> > > > > Caravel is nice visualization tool recently open sourced by airbnb.
> > Did
> > > > > anyone try to integrate Drill and/or interested in contributing to
> > > making
> > > > > this work with Drill.
> > > > >
> > > > > https://github.com/airbnb/caravel
> > > > >
> > > > >
> > > > > -Thanks
> > > > > Neeraja
> > > > >
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by Neeraja Rentachintala <nr...@maprtech.com>.
Yes, the key thing is the SQL Alchemy layer.
I can see it more broadly being used than just Caravel.
On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com> wrote:
> Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
> enable other frameworks.
>
> Would one think that adaptation of SQLAlchemy to Drill is specific to
> Caravel? What subset of features from a RDBMS ORM is meaningfull, feasable
> and usefull to map to Drill. This sounds like a broad general question. I
> am sure there are orms from other language camps that might want Drill
> backends.
> On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
>
> > I will be looking into this as well, thanks for sharing!
> > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com> wrote:
> >
> > > I Hi Neeraja,
> > >
> > > I am interested in contributing if integration is not available.
> > > Kindly let me know
> > >
> > > Regards,
> > > Nirav
> > >
> > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > nrentachintala@maprtech.com> wrote:
> > >
> > > > Hi Folks
> > > >
> > > > Caravel is nice visualization tool recently open sourced by airbnb.
> Did
> > > > anyone try to integrate Drill and/or interested in contributing to
> > making
> > > > this work with Drill.
> > > >
> > > > https://github.com/airbnb/caravel
> > > >
> > > >
> > > > -Thanks
> > > > Neeraja
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
I've updated the https://github.com/JohnOmernik/caraveldrill
At this point, adding a database with Drill works! You can now press test
connection and we get a "Seems Ok" As we prune out old stuff, and add in
Drill stuff, I am using comments to identify what we added and what has
been there, if a class in the dialect has no comments, assume it's all
MSAccess dialect code... (I know this is a horrible way to code, but I am
learning and like sharing my learnings :)
John
On Mon, May 16, 2016 at 2:26 PM, John Omernik <jo...@omernik.com> wrote:
> AWESOME! Yep, that works and I like it better than using sys.options. Now
> to dive in and play with the Dialect.
>
> Note, I will be uploading an unfinished dialect in my caraveldrill repo...
> the goal isn't a production ready thing, but a skeleton (based on the
> access one) of what a dialect is... The purpose is to evolve things as put
> it all in one area for people to work with. As of now, I have no clue how
> to trap statements with no FROM, but that's my first thing to work on :)
>
> Thanks for the help Veera!
>
> John
>
> On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
> vnaranammalpuram@maprtech.com> wrote:
>
>> Does this work?
>>
>> 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
>> +-------------+
>> | some_label |
>> +-------------+
>> | x |
>> +-------------+
>> 1 row selected (1.41 seconds)
>> 0: jdbc:drill:zk=local>
>>
>> -Veera
>>
>> On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > I suppose I could do select 'x' AS some_label from sys.options limit 1;
>> >
>> > any reason not to? Any other options?
>> >
>> > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com> wrote:
>> >
>> > > Does Drill have a "dummy" table (like dual) that we could test
>> against?
>> > If
>> > > we had that I could replace the that in a dialect (I think)
>> > >
>> > > SELECT 'test plain returns' AS anon_1
>> > >
>> > >
>> > > SELECT 'x' AS some_label
>> > >
>> > >
>> > > SELECT 'test unicode returns' AS anon_1
>> > >
>> > >
>> > > SELECT 'x' AS some_label
>> > >
>> > >
>> > > Drill is looking for a "FROM" :)
>> > >
>> > >
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Veera Naranammalpuram
>> Product Specialist - SQL on Hadoop
>> *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
>> *(Email) vnaranammalpuram@maprtech.com <na...@maprtech.com>*
>> *(Mobile) 917 683 8116 - can text *
>> *Timezone: ET (UTC -5:00 / -4:00)*
>>
>
>
Re: Drill & Caravel
Posted by Boris Chmiel <bo...@yahoo.com.INVALID>.
Super interesting
Le Mardi 17 mai 2016 14h56, John Omernik <jo...@omernik.com> a écrit :
So in addition to the docker env to play with drill and caravel at
https://github.com/JohnOmernik/caraveldrill
We also have a Dialect going at:
https://github.com/JohnOmernik/sqlalchemy-drill
Now, this dialect doesn't do much but allow a connect using pyodbc, thus
it's not something where you can dive in and use. Basically we are using
it, based on a pyodbc access dialect to understand dialects and update
pieces (and remove access pieces) as they cause issue for Caravel...it's an
iterative approach... but it's how I learn so if your approach is to
understand SQLAlchemy and write from scratch, this is not the Repo(s) for
you...
Right now we have in the caraveldrill repo:
- Working Docker image that loads all the uniodbc, mapr odbc, caravel, pip,
and caravel prereqs
- It updates settings so MapR ODBC can talk to drill with the proper unicode
- It installs caravel, has that working
- It has a sample python script that validates that outside of sql alchemy
and caravel, your settings for pyodbc are working, and also provides
helpful connection strings for caravel
- It DOES NOT install the sqlalchemy dialect in the sqlalchemy-drill repo.
You have to clone that and install it in the container yourself using
python setup.py install.
- Once that is done, the connection test will work successfully, and now
it's hacking on the dialect until we see pretty visualizations :)
John
On Mon, May 16, 2016 at 4:09 PM, John Omernik <jo...@omernik.com> wrote:
> Ya happy to join the hangout tomorrow.
>
> On Mon, May 16, 2016 at 3:09 PM, Neeraja Rentachintala <
> nrentachintala@maprtech.com> wrote:
>
>> John
>> Great. Can we briefly look at this during hang out tomorrow.
>>
>> On Mon, May 16, 2016 at 12:26 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > AWESOME! Yep, that works and I like it better than using sys.options.
>> Now
>> > to dive in and play with the Dialect.
>> >
>> > Note, I will be uploading an unfinished dialect in my caraveldrill
>> repo...
>> > the goal isn't a production ready thing, but a skeleton (based on the
>> > access one) of what a dialect is... The purpose is to evolve things as
>> put
>> > it all in one area for people to work with. As of now, I have no clue
>> how
>> > to trap statements with no FROM, but that's my first thing to work on :)
>> >
>> > Thanks for the help Veera!
>> >
>> > John
>> >
>> > On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
>> > vnaranammalpuram@maprtech.com> wrote:
>> >
>> > > Does this work?
>> > >
>> > > 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
>> > > +-------------+
>> > > | some_label |
>> > > +-------------+
>> > > | x |
>> > > +-------------+
>> > > 1 row selected (1.41 seconds)
>> > > 0: jdbc:drill:zk=local>
>> > >
>> > > -Veera
>> > >
>> > > On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com>
>> wrote:
>> > >
>> > > > I suppose I could do select 'x' AS some_label from sys.options
>> limit 1;
>> > > >
>> > > > any reason not to? Any other options?
>> > > >
>> > > > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com>
>> > wrote:
>> > > >
>> > > > > Does Drill have a "dummy" table (like dual) that we could test
>> > against?
>> > > > If
>> > > > > we had that I could replace the that in a dialect (I think)
>> > > > >
>> > > > > SELECT 'test plain returns' AS anon_1
>> > > > >
>> > > > >
>> > > > > SELECT 'x' AS some_label
>> > > > >
>> > > > >
>> > > > > SELECT 'test unicode returns' AS anon_1
>> > > > >
>> > > > >
>> > > > > SELECT 'x' AS some_label
>> > > > >
>> > > > >
>> > > > > Drill is looking for a "FROM" :)
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Veera Naranammalpuram
>> > > Product Specialist - SQL on Hadoop
>> > > *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
>> > > *(Email) vnaranammalpuram@maprtech.com <naranammalpuram@maprtech.com
>> >*
>> > > *(Mobile) 917 683 8116 - can text *
>> > > *Timezone: ET (UTC -5:00 / -4:00)*
>> > >
>> >
>>
>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
So in addition to the docker env to play with drill and caravel at
https://github.com/JohnOmernik/caraveldrill
We also have a Dialect going at:
https://github.com/JohnOmernik/sqlalchemy-drill
Now, this dialect doesn't do much but allow a connect using pyodbc, thus
it's not something where you can dive in and use. Basically we are using
it, based on a pyodbc access dialect to understand dialects and update
pieces (and remove access pieces) as they cause issue for Caravel...it's an
iterative approach... but it's how I learn so if your approach is to
understand SQLAlchemy and write from scratch, this is not the Repo(s) for
you...
Right now we have in the caraveldrill repo:
- Working Docker image that loads all the uniodbc, mapr odbc, caravel, pip,
and caravel prereqs
- It updates settings so MapR ODBC can talk to drill with the proper unicode
- It installs caravel, has that working
- It has a sample python script that validates that outside of sql alchemy
and caravel, your settings for pyodbc are working, and also provides
helpful connection strings for caravel
- It DOES NOT install the sqlalchemy dialect in the sqlalchemy-drill repo.
You have to clone that and install it in the container yourself using
python setup.py install.
- Once that is done, the connection test will work successfully, and now
it's hacking on the dialect until we see pretty visualizations :)
John
On Mon, May 16, 2016 at 4:09 PM, John Omernik <jo...@omernik.com> wrote:
> Ya happy to join the hangout tomorrow.
>
> On Mon, May 16, 2016 at 3:09 PM, Neeraja Rentachintala <
> nrentachintala@maprtech.com> wrote:
>
>> John
>> Great. Can we briefly look at this during hang out tomorrow.
>>
>> On Mon, May 16, 2016 at 12:26 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > AWESOME! Yep, that works and I like it better than using sys.options.
>> Now
>> > to dive in and play with the Dialect.
>> >
>> > Note, I will be uploading an unfinished dialect in my caraveldrill
>> repo...
>> > the goal isn't a production ready thing, but a skeleton (based on the
>> > access one) of what a dialect is... The purpose is to evolve things as
>> put
>> > it all in one area for people to work with. As of now, I have no clue
>> how
>> > to trap statements with no FROM, but that's my first thing to work on :)
>> >
>> > Thanks for the help Veera!
>> >
>> > John
>> >
>> > On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
>> > vnaranammalpuram@maprtech.com> wrote:
>> >
>> > > Does this work?
>> > >
>> > > 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
>> > > +-------------+
>> > > | some_label |
>> > > +-------------+
>> > > | x |
>> > > +-------------+
>> > > 1 row selected (1.41 seconds)
>> > > 0: jdbc:drill:zk=local>
>> > >
>> > > -Veera
>> > >
>> > > On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com>
>> wrote:
>> > >
>> > > > I suppose I could do select 'x' AS some_label from sys.options
>> limit 1;
>> > > >
>> > > > any reason not to? Any other options?
>> > > >
>> > > > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com>
>> > wrote:
>> > > >
>> > > > > Does Drill have a "dummy" table (like dual) that we could test
>> > against?
>> > > > If
>> > > > > we had that I could replace the that in a dialect (I think)
>> > > > >
>> > > > > SELECT 'test plain returns' AS anon_1
>> > > > >
>> > > > >
>> > > > > SELECT 'x' AS some_label
>> > > > >
>> > > > >
>> > > > > SELECT 'test unicode returns' AS anon_1
>> > > > >
>> > > > >
>> > > > > SELECT 'x' AS some_label
>> > > > >
>> > > > >
>> > > > > Drill is looking for a "FROM" :)
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Veera Naranammalpuram
>> > > Product Specialist - SQL on Hadoop
>> > > *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
>> > > *(Email) vnaranammalpuram@maprtech.com <naranammalpuram@maprtech.com
>> >*
>> > > *(Mobile) 917 683 8116 - can text *
>> > > *Timezone: ET (UTC -5:00 / -4:00)*
>> > >
>> >
>>
>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
Ya happy to join the hangout tomorrow.
On Mon, May 16, 2016 at 3:09 PM, Neeraja Rentachintala <
nrentachintala@maprtech.com> wrote:
> John
> Great. Can we briefly look at this during hang out tomorrow.
>
> On Mon, May 16, 2016 at 12:26 PM, John Omernik <jo...@omernik.com> wrote:
>
> > AWESOME! Yep, that works and I like it better than using sys.options. Now
> > to dive in and play with the Dialect.
> >
> > Note, I will be uploading an unfinished dialect in my caraveldrill
> repo...
> > the goal isn't a production ready thing, but a skeleton (based on the
> > access one) of what a dialect is... The purpose is to evolve things as
> put
> > it all in one area for people to work with. As of now, I have no clue how
> > to trap statements with no FROM, but that's my first thing to work on :)
> >
> > Thanks for the help Veera!
> >
> > John
> >
> > On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
> > vnaranammalpuram@maprtech.com> wrote:
> >
> > > Does this work?
> > >
> > > 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
> > > +-------------+
> > > | some_label |
> > > +-------------+
> > > | x |
> > > +-------------+
> > > 1 row selected (1.41 seconds)
> > > 0: jdbc:drill:zk=local>
> > >
> > > -Veera
> > >
> > > On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > > > I suppose I could do select 'x' AS some_label from sys.options limit
> 1;
> > > >
> > > > any reason not to? Any other options?
> > > >
> > > > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com>
> > wrote:
> > > >
> > > > > Does Drill have a "dummy" table (like dual) that we could test
> > against?
> > > > If
> > > > > we had that I could replace the that in a dialect (I think)
> > > > >
> > > > > SELECT 'test plain returns' AS anon_1
> > > > >
> > > > >
> > > > > SELECT 'x' AS some_label
> > > > >
> > > > >
> > > > > SELECT 'test unicode returns' AS anon_1
> > > > >
> > > > >
> > > > > SELECT 'x' AS some_label
> > > > >
> > > > >
> > > > > Drill is looking for a "FROM" :)
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Veera Naranammalpuram
> > > Product Specialist - SQL on Hadoop
> > > *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
> > > *(Email) vnaranammalpuram@maprtech.com <na...@maprtech.com>*
> > > *(Mobile) 917 683 8116 - can text *
> > > *Timezone: ET (UTC -5:00 / -4:00)*
> > >
> >
>
Re: Drill & Caravel
Posted by Neeraja Rentachintala <nr...@maprtech.com>.
John
Great. Can we briefly look at this during hang out tomorrow.
On Mon, May 16, 2016 at 12:26 PM, John Omernik <jo...@omernik.com> wrote:
> AWESOME! Yep, that works and I like it better than using sys.options. Now
> to dive in and play with the Dialect.
>
> Note, I will be uploading an unfinished dialect in my caraveldrill repo...
> the goal isn't a production ready thing, but a skeleton (based on the
> access one) of what a dialect is... The purpose is to evolve things as put
> it all in one area for people to work with. As of now, I have no clue how
> to trap statements with no FROM, but that's my first thing to work on :)
>
> Thanks for the help Veera!
>
> John
>
> On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
> vnaranammalpuram@maprtech.com> wrote:
>
> > Does this work?
> >
> > 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
> > +-------------+
> > | some_label |
> > +-------------+
> > | x |
> > +-------------+
> > 1 row selected (1.41 seconds)
> > 0: jdbc:drill:zk=local>
> >
> > -Veera
> >
> > On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com> wrote:
> >
> > > I suppose I could do select 'x' AS some_label from sys.options limit 1;
> > >
> > > any reason not to? Any other options?
> > >
> > > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > > > Does Drill have a "dummy" table (like dual) that we could test
> against?
> > > If
> > > > we had that I could replace the that in a dialect (I think)
> > > >
> > > > SELECT 'test plain returns' AS anon_1
> > > >
> > > >
> > > > SELECT 'x' AS some_label
> > > >
> > > >
> > > > SELECT 'test unicode returns' AS anon_1
> > > >
> > > >
> > > > SELECT 'x' AS some_label
> > > >
> > > >
> > > > Drill is looking for a "FROM" :)
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Veera Naranammalpuram
> > Product Specialist - SQL on Hadoop
> > *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
> > *(Email) vnaranammalpuram@maprtech.com <na...@maprtech.com>*
> > *(Mobile) 917 683 8116 - can text *
> > *Timezone: ET (UTC -5:00 / -4:00)*
> >
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
AWESOME! Yep, that works and I like it better than using sys.options. Now
to dive in and play with the Dialect.
Note, I will be uploading an unfinished dialect in my caraveldrill repo...
the goal isn't a production ready thing, but a skeleton (based on the
access one) of what a dialect is... The purpose is to evolve things as put
it all in one area for people to work with. As of now, I have no clue how
to trap statements with no FROM, but that's my first thing to work on :)
Thanks for the help Veera!
John
On Mon, May 16, 2016 at 2:20 PM, Veera Naranammalpuram <
vnaranammalpuram@maprtech.com> wrote:
> Does this work?
>
> 0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
> +-------------+
> | some_label |
> +-------------+
> | x |
> +-------------+
> 1 row selected (1.41 seconds)
> 0: jdbc:drill:zk=local>
>
> -Veera
>
> On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com> wrote:
>
> > I suppose I could do select 'x' AS some_label from sys.options limit 1;
> >
> > any reason not to? Any other options?
> >
> > On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com> wrote:
> >
> > > Does Drill have a "dummy" table (like dual) that we could test against?
> > If
> > > we had that I could replace the that in a dialect (I think)
> > >
> > > SELECT 'test plain returns' AS anon_1
> > >
> > >
> > > SELECT 'x' AS some_label
> > >
> > >
> > > SELECT 'test unicode returns' AS anon_1
> > >
> > >
> > > SELECT 'x' AS some_label
> > >
> > >
> > > Drill is looking for a "FROM" :)
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> Veera Naranammalpuram
> Product Specialist - SQL on Hadoop
> *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
> *(Email) vnaranammalpuram@maprtech.com <na...@maprtech.com>*
> *(Mobile) 917 683 8116 - can text *
> *Timezone: ET (UTC -5:00 / -4:00)*
>
Re: Drill & Caravel
Posted by Veera Naranammalpuram <vn...@maprtech.com>.
Does this work?
0: jdbc:drill:zk=local> SELECT 'x' AS some_label from (values(1));
+-------------+
| some_label |
+-------------+
| x |
+-------------+
1 row selected (1.41 seconds)
0: jdbc:drill:zk=local>
-Veera
On Mon, May 16, 2016 at 3:19 PM, John Omernik <jo...@omernik.com> wrote:
> I suppose I could do select 'x' AS some_label from sys.options limit 1;
>
> any reason not to? Any other options?
>
> On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com> wrote:
>
> > Does Drill have a "dummy" table (like dual) that we could test against?
> If
> > we had that I could replace the that in a dialect (I think)
> >
> > SELECT 'test plain returns' AS anon_1
> >
> >
> > SELECT 'x' AS some_label
> >
> >
> > SELECT 'test unicode returns' AS anon_1
> >
> >
> > SELECT 'x' AS some_label
> >
> >
> > Drill is looking for a "FROM" :)
> >
> >
> >
> >
>
--
Veera Naranammalpuram
Product Specialist - SQL on Hadoop
*MapR Technologies (www.mapr.com <http://www.mapr.com>)*
*(Email) vnaranammalpuram@maprtech.com <na...@maprtech.com>*
*(Mobile) 917 683 8116 - can text *
*Timezone: ET (UTC -5:00 / -4:00)*
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
I suppose I could do select 'x' AS some_label from sys.options limit 1;
any reason not to? Any other options?
On Mon, May 16, 2016 at 2:18 PM, John Omernik <jo...@omernik.com> wrote:
> Does Drill have a "dummy" table (like dual) that we could test against? If
> we had that I could replace the that in a dialect (I think)
>
> SELECT 'test plain returns' AS anon_1
>
>
> SELECT 'x' AS some_label
>
>
> SELECT 'test unicode returns' AS anon_1
>
>
> SELECT 'x' AS some_label
>
>
> Drill is looking for a "FROM" :)
>
>
>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
Does Drill have a "dummy" table (like dual) that we could test against? If
we had that I could replace the that in a dialect (I think)
SELECT 'test plain returns' AS anon_1
SELECT 'x' AS some_label
SELECT 'test unicode returns' AS anon_1
SELECT 'x' AS some_label
Drill is looking for a "FROM" :)
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
I managed to get a real sql statement from Caravel to Drill ! It failed,
but at least it wasn't a blob of Unicode text. I had to switch UTF-32 to
UTF-16 in mapr.drillodbc.ini. I now do this directly in my dockerfile for
caraveldrill https://github.com/JohnOmernik/caraveldrill
Now I need to look at the SQL Alchemy side of things. Drill didn't like
some of the queries that got sent when I did test connections etc. So now
to figure out how to handle that in the dialect...
(Example of Queries Drill didn't like that Caravel/SQLAlchemy Sent when
adding data connections)
SELECT 'test plain returns' AS anon_1
SELECT 'x' AS some_label
SELECT 'test unicode returns' AS anon_1
SELECT 'x' AS some_label
On Sun, May 15, 2016 at 2:30 PM, John Omernik <jo...@omernik.com> wrote:
> I managed to get a skeleton drill dialect to connect to Drill. That said,
> when I "skeletonized" the access Dialect I likely removed to much, in that
> when I tested the connection from Caravel to Drill, this is the error I got
> (note the query was attempted and I saw it in my drill profiles!!)
>
> Basically I used the URL drill+pyodbc:///?%CONNSTRING% wtih %CONNSTRING
> being the uq variable in my pyodbc_example.py that I outputted in my github
> above. That worked except that I needed to set
>
> {
> "metadata_params": {},
> "engine_params": {"connect_args": {"autocommit":1}}
> }
>
> in the Caravel data source page as well as the URL to make it work.
>
> Anywho, I think I am out of my depth at this point, I got it to connect
> and submit "something" to drill, but that's about it... if anyone needs
> help getting to where I got to take it from there, I'd be happy to help.
>
> John
>
>
>
> DBAPIError: (pyodbc.Error) ('HY000', '[HY000] [MapR][Drill] (1040) Drill
> failed to execute the query:
> \x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\n[30027]Query
> execution error. Details:[ \nPARSE ERROR: Lexical error at line 1, column
> 1. Encountered: "\\ufffd" (65533), after : ""\n\nSQL Query
> \x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\n^\n\n\n[Error
> Id: 9e67eab5-3a39-4527-ab22-763e950dcd86 on
> hadoopmapr5.brewingintel.com:20001]\n] (1040) (SQLExecDirectW)')
>
> On Sun, May 15, 2016 at 10:24 AM, John Omernik <jo...@omernik.com> wrote:
>
>> Based on some of the SQL Alchemy documentation, they recommend using the
>> pyodbc-access dialect as a good starting point. (Fairly simple, easy to
>> understand how things play out)
>>
>> https://bitbucket.org/zzzeek/sqlalchemy-access
>>
>> I am looking through the code, but my time to dedicate to this is limited
>> do to an implementation at work that is finally happening. I will continue
>> to post updates as come across them.
>>
>> As a side, I created a git with a simple dev env for Caravel and pyodbc
>> built in a docker container. This includes the MapR ODBC driver, the unix
>> ODBC, and pyodbc. (And has an example python script showing the connection
>> to drill works).
>>
>> Basically, it's everything except the SQL Alchemy Dialect work, Caravel
>> is working with the test data. (And it's persistent, so when you shut down
>> the container, it actually persists the setup work in caravel, so the next
>> time you start, you can just work again). It's not production ready, but
>> it allows people to play around with things, and shows off using python
>> with Drill via pyodbc, using caravel, and allows folks who may know sql
>> alchemy a starting off point.
>>
>> https://github.com/JohnOmernik/caraveldrill
>>
>>
>> John
>>
>>
>>
>>
>> On Sun, May 15, 2016 at 12:34 AM, Ted Dunning <te...@gmail.com>
>> wrote:
>>
>>> On Sat, May 14, 2016 at 6:31 PM, John Omernik <jo...@omernik.com> wrote:
>>>
>>> > Thoughts on approaching writing a dialect for Drill for SQL alchemy?
>>> > Anyone here done that before?
>>> >
>>>
>>> Nobody has. It is moderately involved, but doesn't look complicated ...
>>> just lengthy.
>>>
>>> Neeraja is coordinating efforts on this.
>>>
>>
>>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
I managed to get a skeleton drill dialect to connect to Drill. That said,
when I "skeletonized" the access Dialect I likely removed to much, in that
when I tested the connection from Caravel to Drill, this is the error I got
(note the query was attempted and I saw it in my drill profiles!!)
Basically I used the URL drill+pyodbc:///?%CONNSTRING% wtih %CONNSTRING
being the uq variable in my pyodbc_example.py that I outputted in my github
above. That worked except that I needed to set
{
"metadata_params": {},
"engine_params": {"connect_args": {"autocommit":1}}
}
in the Caravel data source page as well as the URL to make it work.
Anywho, I think I am out of my depth at this point, I got it to connect and
submit "something" to drill, but that's about it... if anyone needs help
getting to where I got to take it from there, I'd be happy to help.
John
DBAPIError: (pyodbc.Error) ('HY000', '[HY000] [MapR][Drill] (1040) Drill
failed to execute the query:
\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\n[30027]Query
execution error. Details:[ \nPARSE ERROR: Lexical error at line 1, column
1. Encountered: "\\ufffd" (65533), after : ""\n\nSQL Query
\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\x1a\n^\n\n\n[Error
Id: 9e67eab5-3a39-4527-ab22-763e950dcd86 on
hadoopmapr5.brewingintel.com:20001]\n] (1040) (SQLExecDirectW)')
On Sun, May 15, 2016 at 10:24 AM, John Omernik <jo...@omernik.com> wrote:
> Based on some of the SQL Alchemy documentation, they recommend using the
> pyodbc-access dialect as a good starting point. (Fairly simple, easy to
> understand how things play out)
>
> https://bitbucket.org/zzzeek/sqlalchemy-access
>
> I am looking through the code, but my time to dedicate to this is limited
> do to an implementation at work that is finally happening. I will continue
> to post updates as come across them.
>
> As a side, I created a git with a simple dev env for Caravel and pyodbc
> built in a docker container. This includes the MapR ODBC driver, the unix
> ODBC, and pyodbc. (And has an example python script showing the connection
> to drill works).
>
> Basically, it's everything except the SQL Alchemy Dialect work, Caravel is
> working with the test data. (And it's persistent, so when you shut down the
> container, it actually persists the setup work in caravel, so the next time
> you start, you can just work again). It's not production ready, but it
> allows people to play around with things, and shows off using python with
> Drill via pyodbc, using caravel, and allows folks who may know sql alchemy
> a starting off point.
>
> https://github.com/JohnOmernik/caraveldrill
>
>
> John
>
>
>
>
> On Sun, May 15, 2016 at 12:34 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
>> On Sat, May 14, 2016 at 6:31 PM, John Omernik <jo...@omernik.com> wrote:
>>
>> > Thoughts on approaching writing a dialect for Drill for SQL alchemy?
>> > Anyone here done that before?
>> >
>>
>> Nobody has. It is moderately involved, but doesn't look complicated ...
>> just lengthy.
>>
>> Neeraja is coordinating efforts on this.
>>
>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
Based on some of the SQL Alchemy documentation, they recommend using the
pyodbc-access dialect as a good starting point. (Fairly simple, easy to
understand how things play out)
https://bitbucket.org/zzzeek/sqlalchemy-access
I am looking through the code, but my time to dedicate to this is limited
do to an implementation at work that is finally happening. I will continue
to post updates as come across them.
As a side, I created a git with a simple dev env for Caravel and pyodbc
built in a docker container. This includes the MapR ODBC driver, the unix
ODBC, and pyodbc. (And has an example python script showing the connection
to drill works).
Basically, it's everything except the SQL Alchemy Dialect work, Caravel is
working with the test data. (And it's persistent, so when you shut down the
container, it actually persists the setup work in caravel, so the next time
you start, you can just work again). It's not production ready, but it
allows people to play around with things, and shows off using python with
Drill via pyodbc, using caravel, and allows folks who may know sql alchemy
a starting off point.
https://github.com/JohnOmernik/caraveldrill
John
On Sun, May 15, 2016 at 12:34 AM, Ted Dunning <te...@gmail.com> wrote:
> On Sat, May 14, 2016 at 6:31 PM, John Omernik <jo...@omernik.com> wrote:
>
> > Thoughts on approaching writing a dialect for Drill for SQL alchemy?
> > Anyone here done that before?
> >
>
> Nobody has. It is moderately involved, but doesn't look complicated ...
> just lengthy.
>
> Neeraja is coordinating efforts on this.
>
Re: Drill & Caravel
Posted by Ted Dunning <te...@gmail.com>.
On Sat, May 14, 2016 at 6:31 PM, John Omernik <jo...@omernik.com> wrote:
> Thoughts on approaching writing a dialect for Drill for SQL alchemy?
> Anyone here done that before?
>
Nobody has. It is moderately involved, but doesn't look complicated ...
just lengthy.
Neeraja is coordinating efforts on this.
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
Hey Chris, Thus far I've managed to get the MapR ODBC driver working in my
Ubuntu container (unpack the redhat RPM and it works with some tweaking) as
well as getting caravel running. Where I am struggling is with Sql Alchemy
itself. I thought dialects would be easy to copy and paste, but they are
much more involved that I would have thought. I tried just applying a
MySQL dialect and it doesn't like it. while I can get data working with no
issue in python via pyodbc, I am struggling with the next steps. To me
getting a good drill dialect that can be used for Sql Alchemy would solve
that problem, however, I need to explore more if it's something I can
devote some time too.
Note, based on my other thread, the MapR ODBC driver has a bug (I think)
related to the zk.root. If you specify a different zk.root in your
drill-override, the ODBC doesn't appear to have (or is not well documented)
((or worst: it is well documented and I missed it) a way to use a different
zk.root, and insteal defaults to /drill. So if I have a zk cluster named
mybits and a drill root of drillprod, I can't access it via zookeeper and
odbc (I can do direct).
Thoughts on approaching writing a dialect for Drill for SQL alchemy?
Anyone here done that before?
John
On Fri, May 13, 2016 at 3:48 PM, Christopher Matta <ch...@mapr.com> wrote:
> John, I've got some sample working code in this notebook for connecting to
> Drill via pyodbc: https://github.com/cjmatta/DrillPandasReddit
>
> I'd be interested in helping integrate this as well. Let us know if you
> push out a repo.
>
> --
> Chris Matta
> 215-701-3146
> chris@mapr.com
>
> On Fri, May 13, 2016 at 2:28 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > The limit 0 form is highly to be preferred since that will give you
> > information about columns for a view.
> >
> > This can be set in the dialect for SQL Alchemy.
> >
> >
> >
> > On Fri, May 13, 2016 at 11:03 AM, Neeraja Rentachintala <
> > nrentachintala@maprtech.com> wrote:
> >
> > > Great, thanks John. I wil look forward for an update on how drill
> queuing
> > > part goes : )
> > > Btw with regards to metadata queries, Drill already supports metadata
> > (both
> > > the limit 0 form and also show tables/show schemas which are served
> from
> > > the Information_schema).
> > >
> > > On Friday, May 13, 2016, John Omernik <jo...@omernik.com> wrote:
> > >
> > > > So with that Docker file, I got caravel working easily with test data
> > (no
> > > > drill yet) that will be weekend fun (and the pyodbc is already
> > installed
> > > in
> > > > the container, so now it's time to play!)
> > > >
> > > > So I started my docker image with:
> > > >
> > > > sudo docker run -it --rm --net=host
> > > > -v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw
> > > > -v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel
> > > > /bin/bash
> > > >
> > > >
> > > > Now, I passed through a couple of volumes that I am not sure I will
> > > need, I
> > > > want to play so that my "State" and initialization is saved in those
> > > > directories in the running container (this is just early testing) I
> > just
> > > > run bash, and then run the commands below and it works. I was lazy
> here
> > > and
> > > > just did net host, it would likely work with bridged mode, but I am
> in
> > an
> > > > airport and wanted to see if I could get it working... the fun part
> > will
> > > be
> > > > working with Drill over the weekend. Thanks again Neeraja for sharing
> > > this!
> > > >
> > > >
> > > >
> > > >
> > > > Then I ran these commands(per the docs) and could explore... pretty
> > easy
> > > > actually!
> > > >
> > > > # Create an admin userfabmanager create-admin --app caravel
> > > > # Initialize the databasecaravel db upgrade
> > > > # Create default roles and permissionscaravel init
> > > > # Load some data to play withcaravel load_examples
> > > > # Start the development web servercaravel runserver -d
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, May 13, 2016 at 11:27 AM, John Omernik <jo...@omernik.com>
> > wrote:
> > > >
> > > > > So, without running this, but having it build successfully, this
> > seems
> > > > > like a good place to start, it has caravel, and pyodbc all
> installed
> > > > here.
> > > > > I will be playing more this weekend....
> > > > >
> > > > > FROM ubuntu
> > > > >
> > > > > RUN apt-get update && apt-get install -y build-essential libssl-dev
> > > > > libffi-dev python-dev python-pip
> > > > >
> > > > > RUN apt-get install -y unixodbc-dev unixodbc-bin
> > > > >
> > > > > RUN pip install pyodbc
> > > > >
> > > > > RUN pip install caravel
> > > > >
> > > > > CMD ["python -v"]
> > > > >
> > > > > On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com>
> > > wrote:
> > > > >
> > > > >> A little more googling and I found the pyodbc, that looks
> promising.
> > > > >>
> > > > >> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com>
> > > > wrote:
> > > > >>
> > > > >>> "SQL Alchemy already understands Drill" I was just looking for
> > that,
> > > is
> > > > >>> there already some docs/blogs on that? I was going to start there
> > as
> > > > well
> > > > >>> to determine how it worked and then look into the dialect writing
> > and
> > > > see
> > > > >>> how big that project was. I didn't find much on the Drill +
> > Alchemy,
> > > > but I
> > > > >>> am in an airport and I blame wifi gremlins.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <
> > ted.dunning@gmail.com
> > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>>> SQLAlchemy generates SQL queries and passes them on to Drill.
> > Since
> > > > >>>> SQLAlchemy already understands Drill, most of what will be
> needed
> > is
> > > > >>>> slight
> > > > >>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy
> to
> > > get
> > > > >>>> meta-data from views. Tableau does the meta-data discovery
> using
> > > > limit
> > > > >>>> 0
> > > > >>>> queries to get column names. We would hope that similar methods
> > > would
> > > > >>>> work.
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <
> > eantelman@gmail.com
> > > >
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support
> could
> > > > likely
> > > > >>>> > enable other frameworks.
> > > > >>>> >
> > > > >>>> > Would one think that adaptation of SQLAlchemy to Drill is
> > specific
> > > > to
> > > > >>>> > Caravel? What subset of features from a RDBMS ORM is
> > meaningfull,
> > > > >>>> feasable
> > > > >>>> > and usefull to map to Drill. This sounds like a broad general
> > > > >>>> question. I
> > > > >>>> > am sure there are orms from other language camps that might
> want
> > > > Drill
> > > > >>>> > backends.
> > > > >>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com>
> > wrote:
> > > > >>>> >
> > > > >>>> > > I will be looking into this as well, thanks for sharing!
> > > > >>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <
> > nirav.shah@games24x7.com
> > > >
> > > > >>>> wrote:
> > > > >>>> > >
> > > > >>>> > > > I Hi Neeraja,
> > > > >>>> > > >
> > > > >>>> > > > I am interested in contributing if integration is not
> > > available.
> > > > >>>> > > > Kindly let me know
> > > > >>>> > > >
> > > > >>>> > > > Regards,
> > > > >>>> > > > Nirav
> > > > >>>> > > >
> > > > >>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > > >>>> > > > nrentachintala@maprtech.com> wrote:
> > > > >>>> > > >
> > > > >>>> > > > > Hi Folks
> > > > >>>> > > > >
> > > > >>>> > > > > Caravel is nice visualization tool recently open sourced
> > by
> > > > >>>> airbnb.
> > > > >>>> > Did
> > > > >>>> > > > > anyone try to integrate Drill and/or interested in
> > > > contributing
> > > > >>>> to
> > > > >>>> > > making
> > > > >>>> > > > > this work with Drill.
> > > > >>>> > > > >
> > > > >>>> > > > > https://github.com/airbnb/caravel
> > > > >>>> > > > >
> > > > >>>> > > > >
> > > > >>>> > > > > -Thanks
> > > > >>>> > > > > Neeraja
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by Christopher Matta <ch...@mapr.com>.
John, I've got some sample working code in this notebook for connecting to
Drill via pyodbc: https://github.com/cjmatta/DrillPandasReddit
I'd be interested in helping integrate this as well. Let us know if you
push out a repo.
--
Chris Matta
215-701-3146
chris@mapr.com
On Fri, May 13, 2016 at 2:28 PM, Ted Dunning <te...@gmail.com> wrote:
> The limit 0 form is highly to be preferred since that will give you
> information about columns for a view.
>
> This can be set in the dialect for SQL Alchemy.
>
>
>
> On Fri, May 13, 2016 at 11:03 AM, Neeraja Rentachintala <
> nrentachintala@maprtech.com> wrote:
>
> > Great, thanks John. I wil look forward for an update on how drill queuing
> > part goes : )
> > Btw with regards to metadata queries, Drill already supports metadata
> (both
> > the limit 0 form and also show tables/show schemas which are served from
> > the Information_schema).
> >
> > On Friday, May 13, 2016, John Omernik <jo...@omernik.com> wrote:
> >
> > > So with that Docker file, I got caravel working easily with test data
> (no
> > > drill yet) that will be weekend fun (and the pyodbc is already
> installed
> > in
> > > the container, so now it's time to play!)
> > >
> > > So I started my docker image with:
> > >
> > > sudo docker run -it --rm --net=host
> > > -v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw
> > > -v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel
> > > /bin/bash
> > >
> > >
> > > Now, I passed through a couple of volumes that I am not sure I will
> > need, I
> > > want to play so that my "State" and initialization is saved in those
> > > directories in the running container (this is just early testing) I
> just
> > > run bash, and then run the commands below and it works. I was lazy here
> > and
> > > just did net host, it would likely work with bridged mode, but I am in
> an
> > > airport and wanted to see if I could get it working... the fun part
> will
> > be
> > > working with Drill over the weekend. Thanks again Neeraja for sharing
> > this!
> > >
> > >
> > >
> > >
> > > Then I ran these commands(per the docs) and could explore... pretty
> easy
> > > actually!
> > >
> > > # Create an admin userfabmanager create-admin --app caravel
> > > # Initialize the databasecaravel db upgrade
> > > # Create default roles and permissionscaravel init
> > > # Load some data to play withcaravel load_examples
> > > # Start the development web servercaravel runserver -d
> > >
> > >
> > >
> > >
> > > On Fri, May 13, 2016 at 11:27 AM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > > > So, without running this, but having it build successfully, this
> seems
> > > > like a good place to start, it has caravel, and pyodbc all installed
> > > here.
> > > > I will be playing more this weekend....
> > > >
> > > > FROM ubuntu
> > > >
> > > > RUN apt-get update && apt-get install -y build-essential libssl-dev
> > > > libffi-dev python-dev python-pip
> > > >
> > > > RUN apt-get install -y unixodbc-dev unixodbc-bin
> > > >
> > > > RUN pip install pyodbc
> > > >
> > > > RUN pip install caravel
> > > >
> > > > CMD ["python -v"]
> > > >
> > > > On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com>
> > wrote:
> > > >
> > > >> A little more googling and I found the pyodbc, that looks promising.
> > > >>
> > > >> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com>
> > > wrote:
> > > >>
> > > >>> "SQL Alchemy already understands Drill" I was just looking for
> that,
> > is
> > > >>> there already some docs/blogs on that? I was going to start there
> as
> > > well
> > > >>> to determine how it worked and then look into the dialect writing
> and
> > > see
> > > >>> how big that project was. I didn't find much on the Drill +
> Alchemy,
> > > but I
> > > >>> am in an airport and I blame wifi gremlins.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <
> ted.dunning@gmail.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>>> SQLAlchemy generates SQL queries and passes them on to Drill.
> Since
> > > >>>> SQLAlchemy already understands Drill, most of what will be needed
> is
> > > >>>> slight
> > > >>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to
> > get
> > > >>>> meta-data from views. Tableau does the meta-data discovery using
> > > limit
> > > >>>> 0
> > > >>>> queries to get column names. We would hope that similar methods
> > would
> > > >>>> work.
> > > >>>>
> > > >>>>
> > > >>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <
> eantelman@gmail.com
> > >
> > > >>>> wrote:
> > > >>>>
> > > >>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could
> > > likely
> > > >>>> > enable other frameworks.
> > > >>>> >
> > > >>>> > Would one think that adaptation of SQLAlchemy to Drill is
> specific
> > > to
> > > >>>> > Caravel? What subset of features from a RDBMS ORM is
> meaningfull,
> > > >>>> feasable
> > > >>>> > and usefull to map to Drill. This sounds like a broad general
> > > >>>> question. I
> > > >>>> > am sure there are orms from other language camps that might want
> > > Drill
> > > >>>> > backends.
> > > >>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com>
> wrote:
> > > >>>> >
> > > >>>> > > I will be looking into this as well, thanks for sharing!
> > > >>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <
> nirav.shah@games24x7.com
> > >
> > > >>>> wrote:
> > > >>>> > >
> > > >>>> > > > I Hi Neeraja,
> > > >>>> > > >
> > > >>>> > > > I am interested in contributing if integration is not
> > available.
> > > >>>> > > > Kindly let me know
> > > >>>> > > >
> > > >>>> > > > Regards,
> > > >>>> > > > Nirav
> > > >>>> > > >
> > > >>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > >>>> > > > nrentachintala@maprtech.com> wrote:
> > > >>>> > > >
> > > >>>> > > > > Hi Folks
> > > >>>> > > > >
> > > >>>> > > > > Caravel is nice visualization tool recently open sourced
> by
> > > >>>> airbnb.
> > > >>>> > Did
> > > >>>> > > > > anyone try to integrate Drill and/or interested in
> > > contributing
> > > >>>> to
> > > >>>> > > making
> > > >>>> > > > > this work with Drill.
> > > >>>> > > > >
> > > >>>> > > > > https://github.com/airbnb/caravel
> > > >>>> > > > >
> > > >>>> > > > >
> > > >>>> > > > > -Thanks
> > > >>>> > > > > Neeraja
> > > >>>> > > > >
> > > >>>> > > >
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > > >>>
> > > >>>
> > > >>
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by Ted Dunning <te...@gmail.com>.
The limit 0 form is highly to be preferred since that will give you
information about columns for a view.
This can be set in the dialect for SQL Alchemy.
On Fri, May 13, 2016 at 11:03 AM, Neeraja Rentachintala <
nrentachintala@maprtech.com> wrote:
> Great, thanks John. I wil look forward for an update on how drill queuing
> part goes : )
> Btw with regards to metadata queries, Drill already supports metadata (both
> the limit 0 form and also show tables/show schemas which are served from
> the Information_schema).
>
> On Friday, May 13, 2016, John Omernik <jo...@omernik.com> wrote:
>
> > So with that Docker file, I got caravel working easily with test data (no
> > drill yet) that will be weekend fun (and the pyodbc is already installed
> in
> > the container, so now it's time to play!)
> >
> > So I started my docker image with:
> >
> > sudo docker run -it --rm --net=host
> > -v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw
> > -v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel
> > /bin/bash
> >
> >
> > Now, I passed through a couple of volumes that I am not sure I will
> need, I
> > want to play so that my "State" and initialization is saved in those
> > directories in the running container (this is just early testing) I just
> > run bash, and then run the commands below and it works. I was lazy here
> and
> > just did net host, it would likely work with bridged mode, but I am in an
> > airport and wanted to see if I could get it working... the fun part will
> be
> > working with Drill over the weekend. Thanks again Neeraja for sharing
> this!
> >
> >
> >
> >
> > Then I ran these commands(per the docs) and could explore... pretty easy
> > actually!
> >
> > # Create an admin userfabmanager create-admin --app caravel
> > # Initialize the databasecaravel db upgrade
> > # Create default roles and permissionscaravel init
> > # Load some data to play withcaravel load_examples
> > # Start the development web servercaravel runserver -d
> >
> >
> >
> >
> > On Fri, May 13, 2016 at 11:27 AM, John Omernik <jo...@omernik.com> wrote:
> >
> > > So, without running this, but having it build successfully, this seems
> > > like a good place to start, it has caravel, and pyodbc all installed
> > here.
> > > I will be playing more this weekend....
> > >
> > > FROM ubuntu
> > >
> > > RUN apt-get update && apt-get install -y build-essential libssl-dev
> > > libffi-dev python-dev python-pip
> > >
> > > RUN apt-get install -y unixodbc-dev unixodbc-bin
> > >
> > > RUN pip install pyodbc
> > >
> > > RUN pip install caravel
> > >
> > > CMD ["python -v"]
> > >
> > > On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com>
> wrote:
> > >
> > >> A little more googling and I found the pyodbc, that looks promising.
> > >>
> > >> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com>
> > wrote:
> > >>
> > >>> "SQL Alchemy already understands Drill" I was just looking for that,
> is
> > >>> there already some docs/blogs on that? I was going to start there as
> > well
> > >>> to determine how it worked and then look into the dialect writing and
> > see
> > >>> how big that project was. I didn't find much on the Drill + Alchemy,
> > but I
> > >>> am in an airport and I blame wifi gremlins.
> > >>>
> > >>>
> > >>>
> > >>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <ted.dunning@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>> SQLAlchemy generates SQL queries and passes them on to Drill. Since
> > >>>> SQLAlchemy already understands Drill, most of what will be needed is
> > >>>> slight
> > >>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to
> get
> > >>>> meta-data from views. Tableau does the meta-data discovery using
> > limit
> > >>>> 0
> > >>>> queries to get column names. We would hope that similar methods
> would
> > >>>> work.
> > >>>>
> > >>>>
> > >>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <eantelman@gmail.com
> >
> > >>>> wrote:
> > >>>>
> > >>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could
> > likely
> > >>>> > enable other frameworks.
> > >>>> >
> > >>>> > Would one think that adaptation of SQLAlchemy to Drill is specific
> > to
> > >>>> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
> > >>>> feasable
> > >>>> > and usefull to map to Drill. This sounds like a broad general
> > >>>> question. I
> > >>>> > am sure there are orms from other language camps that might want
> > Drill
> > >>>> > backends.
> > >>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
> > >>>> >
> > >>>> > > I will be looking into this as well, thanks for sharing!
> > >>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <nirav.shah@games24x7.com
> >
> > >>>> wrote:
> > >>>> > >
> > >>>> > > > I Hi Neeraja,
> > >>>> > > >
> > >>>> > > > I am interested in contributing if integration is not
> available.
> > >>>> > > > Kindly let me know
> > >>>> > > >
> > >>>> > > > Regards,
> > >>>> > > > Nirav
> > >>>> > > >
> > >>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > >>>> > > > nrentachintala@maprtech.com> wrote:
> > >>>> > > >
> > >>>> > > > > Hi Folks
> > >>>> > > > >
> > >>>> > > > > Caravel is nice visualization tool recently open sourced by
> > >>>> airbnb.
> > >>>> > Did
> > >>>> > > > > anyone try to integrate Drill and/or interested in
> > contributing
> > >>>> to
> > >>>> > > making
> > >>>> > > > > this work with Drill.
> > >>>> > > > >
> > >>>> > > > > https://github.com/airbnb/caravel
> > >>>> > > > >
> > >>>> > > > >
> > >>>> > > > > -Thanks
> > >>>> > > > > Neeraja
> > >>>> > > > >
> > >>>> > > >
> > >>>> > >
> > >>>> >
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> >
>
Drill & Caravel
Posted by Neeraja Rentachintala <nr...@maprtech.com>.
Great, thanks John. I wil look forward for an update on how drill queuing
part goes : )
Btw with regards to metadata queries, Drill already supports metadata (both
the limit 0 form and also show tables/show schemas which are served from
the Information_schema).
On Friday, May 13, 2016, John Omernik <jo...@omernik.com> wrote:
> So with that Docker file, I got caravel working easily with test data (no
> drill yet) that will be weekend fun (and the pyodbc is already installed in
> the container, so now it's time to play!)
>
> So I started my docker image with:
>
> sudo docker run -it --rm --net=host
> -v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw
> -v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel
> /bin/bash
>
>
> Now, I passed through a couple of volumes that I am not sure I will need, I
> want to play so that my "State" and initialization is saved in those
> directories in the running container (this is just early testing) I just
> run bash, and then run the commands below and it works. I was lazy here and
> just did net host, it would likely work with bridged mode, but I am in an
> airport and wanted to see if I could get it working... the fun part will be
> working with Drill over the weekend. Thanks again Neeraja for sharing this!
>
>
>
>
> Then I ran these commands(per the docs) and could explore... pretty easy
> actually!
>
> # Create an admin userfabmanager create-admin --app caravel
> # Initialize the databasecaravel db upgrade
> # Create default roles and permissionscaravel init
> # Load some data to play withcaravel load_examples
> # Start the development web servercaravel runserver -d
>
>
>
>
> On Fri, May 13, 2016 at 11:27 AM, John Omernik <jo...@omernik.com> wrote:
>
> > So, without running this, but having it build successfully, this seems
> > like a good place to start, it has caravel, and pyodbc all installed
> here.
> > I will be playing more this weekend....
> >
> > FROM ubuntu
> >
> > RUN apt-get update && apt-get install -y build-essential libssl-dev
> > libffi-dev python-dev python-pip
> >
> > RUN apt-get install -y unixodbc-dev unixodbc-bin
> >
> > RUN pip install pyodbc
> >
> > RUN pip install caravel
> >
> > CMD ["python -v"]
> >
> > On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com> wrote:
> >
> >> A little more googling and I found the pyodbc, that looks promising.
> >>
> >> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com>
> wrote:
> >>
> >>> "SQL Alchemy already understands Drill" I was just looking for that, is
> >>> there already some docs/blogs on that? I was going to start there as
> well
> >>> to determine how it worked and then look into the dialect writing and
> see
> >>> how big that project was. I didn't find much on the Drill + Alchemy,
> but I
> >>> am in an airport and I blame wifi gremlins.
> >>>
> >>>
> >>>
> >>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <te...@gmail.com>
> >>> wrote:
> >>>
> >>>> SQLAlchemy generates SQL queries and passes them on to Drill. Since
> >>>> SQLAlchemy already understands Drill, most of what will be needed is
> >>>> slight
> >>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
> >>>> meta-data from views. Tableau does the meta-data discovery using
> limit
> >>>> 0
> >>>> queries to get column names. We would hope that similar methods would
> >>>> work.
> >>>>
> >>>>
> >>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could
> likely
> >>>> > enable other frameworks.
> >>>> >
> >>>> > Would one think that adaptation of SQLAlchemy to Drill is specific
> to
> >>>> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
> >>>> feasable
> >>>> > and usefull to map to Drill. This sounds like a broad general
> >>>> question. I
> >>>> > am sure there are orms from other language camps that might want
> Drill
> >>>> > backends.
> >>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
> >>>> >
> >>>> > > I will be looking into this as well, thanks for sharing!
> >>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
> >>>> wrote:
> >>>> > >
> >>>> > > > I Hi Neeraja,
> >>>> > > >
> >>>> > > > I am interested in contributing if integration is not available.
> >>>> > > > Kindly let me know
> >>>> > > >
> >>>> > > > Regards,
> >>>> > > > Nirav
> >>>> > > >
> >>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> >>>> > > > nrentachintala@maprtech.com> wrote:
> >>>> > > >
> >>>> > > > > Hi Folks
> >>>> > > > >
> >>>> > > > > Caravel is nice visualization tool recently open sourced by
> >>>> airbnb.
> >>>> > Did
> >>>> > > > > anyone try to integrate Drill and/or interested in
> contributing
> >>>> to
> >>>> > > making
> >>>> > > > > this work with Drill.
> >>>> > > > >
> >>>> > > > > https://github.com/airbnb/caravel
> >>>> > > > >
> >>>> > > > >
> >>>> > > > > -Thanks
> >>>> > > > > Neeraja
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
So with that Docker file, I got caravel working easily with test data (no
drill yet) that will be weekend fun (and the pyodbc is already installed in
the container, so now it's time to play!)
So I started my docker image with:
sudo docker run -it --rm --net=host
-v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw
-v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel
/bin/bash
Now, I passed through a couple of volumes that I am not sure I will need, I
want to play so that my "State" and initialization is saved in those
directories in the running container (this is just early testing) I just
run bash, and then run the commands below and it works. I was lazy here and
just did net host, it would likely work with bridged mode, but I am in an
airport and wanted to see if I could get it working... the fun part will be
working with Drill over the weekend. Thanks again Neeraja for sharing this!
Then I ran these commands(per the docs) and could explore... pretty easy
actually!
# Create an admin userfabmanager create-admin --app caravel
# Initialize the databasecaravel db upgrade
# Create default roles and permissionscaravel init
# Load some data to play withcaravel load_examples
# Start the development web servercaravel runserver -d
On Fri, May 13, 2016 at 11:27 AM, John Omernik <jo...@omernik.com> wrote:
> So, without running this, but having it build successfully, this seems
> like a good place to start, it has caravel, and pyodbc all installed here.
> I will be playing more this weekend....
>
> FROM ubuntu
>
> RUN apt-get update && apt-get install -y build-essential libssl-dev
> libffi-dev python-dev python-pip
>
> RUN apt-get install -y unixodbc-dev unixodbc-bin
>
> RUN pip install pyodbc
>
> RUN pip install caravel
>
> CMD ["python -v"]
>
> On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com> wrote:
>
>> A little more googling and I found the pyodbc, that looks promising.
>>
>> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com> wrote:
>>
>>> "SQL Alchemy already understands Drill" I was just looking for that, is
>>> there already some docs/blogs on that? I was going to start there as well
>>> to determine how it worked and then look into the dialect writing and see
>>> how big that project was. I didn't find much on the Drill + Alchemy, but I
>>> am in an airport and I blame wifi gremlins.
>>>
>>>
>>>
>>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <te...@gmail.com>
>>> wrote:
>>>
>>>> SQLAlchemy generates SQL queries and passes them on to Drill. Since
>>>> SQLAlchemy already understands Drill, most of what will be needed is
>>>> slight
>>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
>>>> meta-data from views. Tableau does the meta-data discovery using limit
>>>> 0
>>>> queries to get column names. We would hope that similar methods would
>>>> work.
>>>>
>>>>
>>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
>>>> wrote:
>>>>
>>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
>>>> > enable other frameworks.
>>>> >
>>>> > Would one think that adaptation of SQLAlchemy to Drill is specific to
>>>> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
>>>> feasable
>>>> > and usefull to map to Drill. This sounds like a broad general
>>>> question. I
>>>> > am sure there are orms from other language camps that might want Drill
>>>> > backends.
>>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
>>>> >
>>>> > > I will be looking into this as well, thanks for sharing!
>>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
>>>> wrote:
>>>> > >
>>>> > > > I Hi Neeraja,
>>>> > > >
>>>> > > > I am interested in contributing if integration is not available.
>>>> > > > Kindly let me know
>>>> > > >
>>>> > > > Regards,
>>>> > > > Nirav
>>>> > > >
>>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
>>>> > > > nrentachintala@maprtech.com> wrote:
>>>> > > >
>>>> > > > > Hi Folks
>>>> > > > >
>>>> > > > > Caravel is nice visualization tool recently open sourced by
>>>> airbnb.
>>>> > Did
>>>> > > > > anyone try to integrate Drill and/or interested in contributing
>>>> to
>>>> > > making
>>>> > > > > this work with Drill.
>>>> > > > >
>>>> > > > > https://github.com/airbnb/caravel
>>>> > > > >
>>>> > > > >
>>>> > > > > -Thanks
>>>> > > > > Neeraja
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
>>>
>>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
So, without running this, but having it build successfully, this seems like
a good place to start, it has caravel, and pyodbc all installed here. I
will be playing more this weekend....
FROM ubuntu
RUN apt-get update && apt-get install -y build-essential libssl-dev
libffi-dev python-dev python-pip
RUN apt-get install -y unixodbc-dev unixodbc-bin
RUN pip install pyodbc
RUN pip install caravel
CMD ["python -v"]
On Fri, May 13, 2016 at 10:44 AM, John Omernik <jo...@omernik.com> wrote:
> A little more googling and I found the pyodbc, that looks promising.
>
> On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com> wrote:
>
>> "SQL Alchemy already understands Drill" I was just looking for that, is
>> there already some docs/blogs on that? I was going to start there as well
>> to determine how it worked and then look into the dialect writing and see
>> how big that project was. I didn't find much on the Drill + Alchemy, but I
>> am in an airport and I blame wifi gremlins.
>>
>>
>>
>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <te...@gmail.com>
>> wrote:
>>
>>> SQLAlchemy generates SQL queries and passes them on to Drill. Since
>>> SQLAlchemy already understands Drill, most of what will be needed is
>>> slight
>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
>>> meta-data from views. Tableau does the meta-data discovery using limit 0
>>> queries to get column names. We would hope that similar methods would
>>> work.
>>>
>>>
>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
>>> wrote:
>>>
>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
>>> > enable other frameworks.
>>> >
>>> > Would one think that adaptation of SQLAlchemy to Drill is specific to
>>> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
>>> feasable
>>> > and usefull to map to Drill. This sounds like a broad general
>>> question. I
>>> > am sure there are orms from other language camps that might want Drill
>>> > backends.
>>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
>>> >
>>> > > I will be looking into this as well, thanks for sharing!
>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
>>> wrote:
>>> > >
>>> > > > I Hi Neeraja,
>>> > > >
>>> > > > I am interested in contributing if integration is not available.
>>> > > > Kindly let me know
>>> > > >
>>> > > > Regards,
>>> > > > Nirav
>>> > > >
>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
>>> > > > nrentachintala@maprtech.com> wrote:
>>> > > >
>>> > > > > Hi Folks
>>> > > > >
>>> > > > > Caravel is nice visualization tool recently open sourced by
>>> airbnb.
>>> > Did
>>> > > > > anyone try to integrate Drill and/or interested in contributing
>>> to
>>> > > making
>>> > > > > this work with Drill.
>>> > > > >
>>> > > > > https://github.com/airbnb/caravel
>>> > > > >
>>> > > > >
>>> > > > > -Thanks
>>> > > > > Neeraja
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
A little more googling and I found the pyodbc, that looks promising.
On Fri, May 13, 2016 at 10:41 AM, John Omernik <jo...@omernik.com> wrote:
> "SQL Alchemy already understands Drill" I was just looking for that, is
> there already some docs/blogs on that? I was going to start there as well
> to determine how it worked and then look into the dialect writing and see
> how big that project was. I didn't find much on the Drill + Alchemy, but I
> am in an airport and I blame wifi gremlins.
>
>
>
> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
>> SQLAlchemy generates SQL queries and passes them on to Drill. Since
>> SQLAlchemy already understands Drill, most of what will be needed is
>> slight
>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
>> meta-data from views. Tableau does the meta-data discovery using limit 0
>> queries to get column names. We would hope that similar methods would
>> work.
>>
>>
>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
>> wrote:
>>
>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
>> > enable other frameworks.
>> >
>> > Would one think that adaptation of SQLAlchemy to Drill is specific to
>> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
>> feasable
>> > and usefull to map to Drill. This sounds like a broad general question.
>> I
>> > am sure there are orms from other language camps that might want Drill
>> > backends.
>> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
>> >
>> > > I will be looking into this as well, thanks for sharing!
>> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
>> wrote:
>> > >
>> > > > I Hi Neeraja,
>> > > >
>> > > > I am interested in contributing if integration is not available.
>> > > > Kindly let me know
>> > > >
>> > > > Regards,
>> > > > Nirav
>> > > >
>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
>> > > > nrentachintala@maprtech.com> wrote:
>> > > >
>> > > > > Hi Folks
>> > > > >
>> > > > > Caravel is nice visualization tool recently open sourced by
>> airbnb.
>> > Did
>> > > > > anyone try to integrate Drill and/or interested in contributing to
>> > > making
>> > > > > this work with Drill.
>> > > > >
>> > > > > https://github.com/airbnb/caravel
>> > > > >
>> > > > >
>> > > > > -Thanks
>> > > > > Neeraja
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
"SQL Alchemy already understands Drill" I was just looking for that, is
there already some docs/blogs on that? I was going to start there as well
to determine how it worked and then look into the dialect writing and see
how big that project was. I didn't find much on the Drill + Alchemy, but I
am in an airport and I blame wifi gremlins.
On Fri, May 13, 2016 at 10:25 AM, Ted Dunning <te...@gmail.com> wrote:
> SQLAlchemy generates SQL queries and passes them on to Drill. Since
> SQLAlchemy already understands Drill, most of what will be needed is slight
> tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
> meta-data from views. Tableau does the meta-data discovery using limit 0
> queries to get column names. We would hope that similar methods would work.
>
>
> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com>
> wrote:
>
> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
> > enable other frameworks.
> >
> > Would one think that adaptation of SQLAlchemy to Drill is specific to
> > Caravel? What subset of features from a RDBMS ORM is meaningfull,
> feasable
> > and usefull to map to Drill. This sounds like a broad general question. I
> > am sure there are orms from other language camps that might want Drill
> > backends.
> > On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
> >
> > > I will be looking into this as well, thanks for sharing!
> > > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com>
> wrote:
> > >
> > > > I Hi Neeraja,
> > > >
> > > > I am interested in contributing if integration is not available.
> > > > Kindly let me know
> > > >
> > > > Regards,
> > > > Nirav
> > > >
> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > > nrentachintala@maprtech.com> wrote:
> > > >
> > > > > Hi Folks
> > > > >
> > > > > Caravel is nice visualization tool recently open sourced by airbnb.
> > Did
> > > > > anyone try to integrate Drill and/or interested in contributing to
> > > making
> > > > > this work with Drill.
> > > > >
> > > > > https://github.com/airbnb/caravel
> > > > >
> > > > >
> > > > > -Thanks
> > > > > Neeraja
> > > > >
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by Ted Dunning <te...@gmail.com>.
SQLAlchemy generates SQL queries and passes them on to Drill. Since
SQLAlchemy already understands Drill, most of what will be needed is slight
tuning for SQL dialect and providing a mechanism for SQLAlchemy to get
meta-data from views. Tableau does the meta-data discovery using limit 0
queries to get column names. We would hope that similar methods would work.
On Fri, May 13, 2016 at 6:13 AM, Erik Antelman <ea...@gmail.com> wrote:
> Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
> enable other frameworks.
>
> Would one think that adaptation of SQLAlchemy to Drill is specific to
> Caravel? What subset of features from a RDBMS ORM is meaningfull, feasable
> and usefull to map to Drill. This sounds like a broad general question. I
> am sure there are orms from other language camps that might want Drill
> backends.
> On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
>
> > I will be looking into this as well, thanks for sharing!
> > On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com> wrote:
> >
> > > I Hi Neeraja,
> > >
> > > I am interested in contributing if integration is not available.
> > > Kindly let me know
> > >
> > > Regards,
> > > Nirav
> > >
> > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > > nrentachintala@maprtech.com> wrote:
> > >
> > > > Hi Folks
> > > >
> > > > Caravel is nice visualization tool recently open sourced by airbnb.
> Did
> > > > anyone try to integrate Drill and/or interested in contributing to
> > making
> > > > this work with Drill.
> > > >
> > > > https://github.com/airbnb/caravel
> > > >
> > > >
> > > > -Thanks
> > > > Neeraja
> > > >
> > >
> >
>
Re: Drill & Caravel
Posted by Erik Antelman <ea...@gmail.com>.
Isn't this a matter of Drill<->SQLAlchemy. Such a support could likely
enable other frameworks.
Would one think that adaptation of SQLAlchemy to Drill is specific to
Caravel? What subset of features from a RDBMS ORM is meaningfull, feasable
and usefull to map to Drill. This sounds like a broad general question. I
am sure there are orms from other language camps that might want Drill
backends.
On May 13, 2016 7:33 AM, "John Omernik" <jo...@omernik.com> wrote:
> I will be looking into this as well, thanks for sharing!
> On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com> wrote:
>
> > I Hi Neeraja,
> >
> > I am interested in contributing if integration is not available.
> > Kindly let me know
> >
> > Regards,
> > Nirav
> >
> > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> > nrentachintala@maprtech.com> wrote:
> >
> > > Hi Folks
> > >
> > > Caravel is nice visualization tool recently open sourced by airbnb. Did
> > > anyone try to integrate Drill and/or interested in contributing to
> making
> > > this work with Drill.
> > >
> > > https://github.com/airbnb/caravel
> > >
> > >
> > > -Thanks
> > > Neeraja
> > >
> >
>
Re: Drill & Caravel
Posted by John Omernik <jo...@omernik.com>.
I will be looking into this as well, thanks for sharing!
On May 13, 2016 2:01 AM, "Nirav Shah" <ni...@games24x7.com> wrote:
> I Hi Neeraja,
>
> I am interested in contributing if integration is not available.
> Kindly let me know
>
> Regards,
> Nirav
>
> On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
> nrentachintala@maprtech.com> wrote:
>
> > Hi Folks
> >
> > Caravel is nice visualization tool recently open sourced by airbnb. Did
> > anyone try to integrate Drill and/or interested in contributing to making
> > this work with Drill.
> >
> > https://github.com/airbnb/caravel
> >
> >
> > -Thanks
> > Neeraja
> >
>
Re: Drill & Caravel
Posted by Nirav Shah <ni...@games24x7.com>.
I Hi Neeraja,
I am interested in contributing if integration is not available.
Kindly let me know
Regards,
Nirav
On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala <
nrentachintala@maprtech.com> wrote:
> Hi Folks
>
> Caravel is nice visualization tool recently open sourced by airbnb. Did
> anyone try to integrate Drill and/or interested in contributing to making
> this work with Drill.
>
> https://github.com/airbnb/caravel
>
>
> -Thanks
> Neeraja
>