Re: Contributing

[Jacques Nadeau <ja...@apache.org>]

Answers inline...

On Sun, Mar 24, 2013 at 8:09 AM, burakkk <bu...@gmail.com> wrote:
> Thanks for the answers. I searched a little bit. I have some questions
> about it.
>
> 1. In the high level architecture page there are some pluggable sources
> that can be rdbms, mongodb or hadoop etc. But every source has different
> data model. For example when we use mongodb(mongoql), does drill pipe to
> mongodb to running the query or drill actually read the data which mongodb
> store its own way? If the answer is drill send mongodb to get the results,
> there's no need to parse query and prepare a logical plan, right? It's also
> eligiable to RDBMS(Oracle, mysql, postresql..etc), Hive, Hbase.

Drill needs to understand the query before it can decide what should
be sent where.  So logical plan, optimization and physical plan all
need to be generated no matter the storage engine.


>
> 2. Drill is using optiq to prepare the execution plan(cost based optimizer)
> so that it decides to which join method(hash join, merge join..etc) will
> use or what execution plan will be, right? From Apache Drill Plan Syntax
> I've seen some analytical sql such as window frame, RunningAggregate. Is it
> in progress? Can we add these functions RANK, CUBE, LAG, LEAD, ROW_NUMBER?
> They're very good functions when you develop data mining application. Is
> the source code in the https://github.com/julianhyde/optiq.git page?

The analytic functions are in progress.  The first phase of Optiq
integration is at the parser level, not at the optimizer level.  The
second phase will be focused on exploring Optiq integration at the
optimizer level.  The code is available on GitHub.

>
> 3. I can't find the physical plans of code in github repository. For
> example where's the merge join impleatition? How can I find it?

Still in development.
>
> I think I can help you one or more of with these items: DRILL-7, DRILL-10,
> DRILL-17, DRILL-34, DRILL-32, DRILL-49. DRILL-7's priority is blocked. Some
> of these issues are in progress. Where do I begin?

One of the easiest ways to figure out how to contribute is to join the
weekly Google Hangout standup.  Sometimes these things are just easier
to discuss "in person".  Once we've talked about things we can report
back any conclusions here.


>how does Optiq decide the data size fit in
>the memory or when does it use the swap operation?

Optiq doesn't decide this, the physical operator implementation does.