You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Chris Baynes <ch...@contiamo.com> on 2016/12/01 16:59:47 UTC
Building a LogicalPlan
Hi,
We have a use case in which we want to construct queries programmatically
which Drill would then execute.
So far I've been able to configure a Jdbc StorageEngine, and initialize a
LogicalPlan (using the builder) with that. I am having difficulty trying to
configure scan, project, and filters. I know I need to construct a
LogicalOperator, and these are implemented in
org.apache.drill.common.logical.data
However to construct an operator instance I need one or more of:
JSONOptions, NamedExpression, LogicalExpression, FieldReference.
So right now I have something like this, but don't know how to fill in the
missing pieces...
// scan
JSONOptions opts = new JSONOptions(...);
builder.addLogicalOperator(Scan.builder().storageEngine("pg").selection(opts).build());
// project
NamedExpression nex = new NamedExpression(...);
builder.addLogicalOperator(Project.builder().addExpr(nex));
LogicalPlan plan = builder.build();
Is there any documentation yet, similar to what calcite has for the
RelBuilder (https://calcite.apache.org/docs/algebra.html)?
I suppose I could also construct a RelNode and then try to convert that to
a LogicalPlan, would I be missing out on any drill features this way?
Thanks in advance
Re: Building a LogicalPlan
Posted by Paul Rogers <pr...@maprtech.com>.
Have you considered something like JOOQ [1] to generate SQL for you, then just use Drill’s SQL support? I’ve not used JOOQ, but I’ve used similar approaches in past projects.
- Paul
[1] http://www.jooq.org
> On Dec 2, 2016, at 7:20 AM, Aman Sinha <am...@apache.org> wrote:
>
> Unfortunately, the 'LogicalPlan' structure that is created by the
> LogicalPlanBuilder does not go through the full Drill query optimization
> process. You are better off starting with a Calcite Rel and then building
> a Drill logical plan with 'Rels' (e.g DrillFilterRel, DrillProjectRel
> etc.).
>
> On Fri, Dec 2, 2016 at 2:21 AM, Ted Dunning <te...@gmail.com> wrote:
>
>> It might be easier to start by stealing a plan from a normal SQL query and
>> then trying to build that. Not that this is a long-term solution, but it
>> gets you going with the many unstated assumptions about a plan that might
>> be present.
>>
>> That would give you something in JSON, however, rather than Java code.
>> That still might be very useful.
>>
>>
>>
>> On Fri, Dec 2, 2016 at 1:59 AM, Chris Baynes <ch...@contiamo.com> wrote:
>>
>>> Hi,
>>>
>>> We have a use case in which we want to construct queries programmatically
>>> which Drill would then execute.
>>>
>>> So far I've been able to configure a Jdbc StorageEngine, and initialize a
>>> LogicalPlan (using the builder) with that. I am having difficulty trying
>> to
>>> configure scan, project, and filters. I know I need to construct a
>>> LogicalOperator, and these are implemented in
>>> org.apache.drill.common.logical.data
>>>
>>> However to construct an operator instance I need one or more of:
>>> JSONOptions, NamedExpression, LogicalExpression, FieldReference.
>>>
>>> So right now I have something like this, but don't know how to fill in
>> the
>>> missing pieces...
>>>
>>> // scan
>>> JSONOptions opts = new JSONOptions(...);
>>> builder.addLogicalOperator(Scan.builder().storageEngine("
>>> pg").selection(opts).build());
>>>
>>> // project
>>> NamedExpression nex = new NamedExpression(...);
>>> builder.addLogicalOperator(Project.builder().addExpr(nex));
>>>
>>> LogicalPlan plan = builder.build();
>>>
>>> Is there any documentation yet, similar to what calcite has for the
>>> RelBuilder (https://calcite.apache.org/docs/algebra.html)?
>>>
>>> I suppose I could also construct a RelNode and then try to convert that
>> to
>>> a LogicalPlan, would I be missing out on any drill features this way?
>>>
>>> Thanks in advance
>>>
>>
Re: Building a LogicalPlan
Posted by Aman Sinha <am...@apache.org>.
Unfortunately, the 'LogicalPlan' structure that is created by the
LogicalPlanBuilder does not go through the full Drill query optimization
process. You are better off starting with a Calcite Rel and then building
a Drill logical plan with 'Rels' (e.g DrillFilterRel, DrillProjectRel
etc.).
On Fri, Dec 2, 2016 at 2:21 AM, Ted Dunning <te...@gmail.com> wrote:
> It might be easier to start by stealing a plan from a normal SQL query and
> then trying to build that. Not that this is a long-term solution, but it
> gets you going with the many unstated assumptions about a plan that might
> be present.
>
> That would give you something in JSON, however, rather than Java code.
> That still might be very useful.
>
>
>
> On Fri, Dec 2, 2016 at 1:59 AM, Chris Baynes <ch...@contiamo.com> wrote:
>
> > Hi,
> >
> > We have a use case in which we want to construct queries programmatically
> > which Drill would then execute.
> >
> > So far I've been able to configure a Jdbc StorageEngine, and initialize a
> > LogicalPlan (using the builder) with that. I am having difficulty trying
> to
> > configure scan, project, and filters. I know I need to construct a
> > LogicalOperator, and these are implemented in
> > org.apache.drill.common.logical.data
> >
> > However to construct an operator instance I need one or more of:
> > JSONOptions, NamedExpression, LogicalExpression, FieldReference.
> >
> > So right now I have something like this, but don't know how to fill in
> the
> > missing pieces...
> >
> > // scan
> > JSONOptions opts = new JSONOptions(...);
> > builder.addLogicalOperator(Scan.builder().storageEngine("
> > pg").selection(opts).build());
> >
> > // project
> > NamedExpression nex = new NamedExpression(...);
> > builder.addLogicalOperator(Project.builder().addExpr(nex));
> >
> > LogicalPlan plan = builder.build();
> >
> > Is there any documentation yet, similar to what calcite has for the
> > RelBuilder (https://calcite.apache.org/docs/algebra.html)?
> >
> > I suppose I could also construct a RelNode and then try to convert that
> to
> > a LogicalPlan, would I be missing out on any drill features this way?
> >
> > Thanks in advance
> >
>
Re: Building a LogicalPlan
Posted by Ted Dunning <te...@gmail.com>.
It might be easier to start by stealing a plan from a normal SQL query and
then trying to build that. Not that this is a long-term solution, but it
gets you going with the many unstated assumptions about a plan that might
be present.
That would give you something in JSON, however, rather than Java code.
That still might be very useful.
On Fri, Dec 2, 2016 at 1:59 AM, Chris Baynes <ch...@contiamo.com> wrote:
> Hi,
>
> We have a use case in which we want to construct queries programmatically
> which Drill would then execute.
>
> So far I've been able to configure a Jdbc StorageEngine, and initialize a
> LogicalPlan (using the builder) with that. I am having difficulty trying to
> configure scan, project, and filters. I know I need to construct a
> LogicalOperator, and these are implemented in
> org.apache.drill.common.logical.data
>
> However to construct an operator instance I need one or more of:
> JSONOptions, NamedExpression, LogicalExpression, FieldReference.
>
> So right now I have something like this, but don't know how to fill in the
> missing pieces...
>
> // scan
> JSONOptions opts = new JSONOptions(...);
> builder.addLogicalOperator(Scan.builder().storageEngine("
> pg").selection(opts).build());
>
> // project
> NamedExpression nex = new NamedExpression(...);
> builder.addLogicalOperator(Project.builder().addExpr(nex));
>
> LogicalPlan plan = builder.build();
>
> Is there any documentation yet, similar to what calcite has for the
> RelBuilder (https://calcite.apache.org/docs/algebra.html)?
>
> I suppose I could also construct a RelNode and then try to convert that to
> a LogicalPlan, would I be missing out on any drill features this way?
>
> Thanks in advance
>