You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jinfeng Ni <ji...@gmail.com> on 2013/12/10 07:16:07 UTC
Discussion of implicit cast design in drill engine.
Hi Yash / All,
I gave some thoughts about how to implement implicit cast. Here is my
preliminary thoughts.
1 . implicit cast vs explicit cast.
Basically, implicit cast would need leverage explicit cast's
implementation. For instance, given an logical expression : 1 + 3.0, if
the function resolver finds that argument 1 should be implicitly casted
into float4, then, drill code should transform the logical expression into
cast(1 as float4) + 3.0, so that "+" operator will call the add(float4,
float4) implementation.
2. Add implicit cast function call in the logical expression tree.
Currently, drill code uses
FunctionImplementationRegistry.getFunction(FunctionCall)
to get DrillFuncHolder in EvalVisitor.
Your FunctionResolver is used to find the best match in the call of
getFunction(). However, if the best match says implicit cast is required,
it's kind of difficult to let getFunction() do 1) insert the cast function
to the logical expression tree, and 2) get cast's DrillFuncHolder, and 3)
generate the code for the cast function.
We probably should separate the process of adding implicit cast from the
logic of FunctionImplementationRegistry.getFunction() and code generation.
To do that :
- Introduce a new Visitor class ImplicitCastBuilder.
- ImplicitCastBuilder will look similar to EvalVisitor .
ImplicitCastBuilder extends AbstractExprVisitor<*LogicalExpression*,
CodeGenerator<?>, RuntimeException>
- ImplicitCastBuilder should build cast function call in *bottom-up*
way.
- ImplicitCastBuilder will modify logical expression tree, and
replace an argument with a cast function call, if your
*FunctionResolver.getBestMatch() *shows implicit cast is required.
- CodeGenerator.addExpr will call ImplicitCastBuilder to insert the
implicit cast() to logical expression tree .
public HoldingContainer addExpr(LogicalExpression ex, boolean rotate){
// logger.debug("Adding next write {}", ex);
if(rotate) rotateBlock();
*ex = implicitCastBuilder.accept(ex, this);*
return evaluationVisitor.addExpr(ex, this);
}
- EvalVisitor will then do the match() and code generation as before.
( No need to call your *FunctionResolver.getBestMatch() *at this stage,
since all the required implicit cast has been inserted into the logical
expression tree).
For example, let's say we have logical expression tree f1 ( a1, f2( a2, a3))
f1()
/ \
/ \
a1 f2()
/ \
/ \
a2 a3
We may end up with the following logical expression tree, after
ImplicitCastBuilder visit.
f1()
/ \
/ \
a1 cast1()
\
\
f2()
/ \
/ \
a2 cast2()
|
|
a3
Note that cast2 would be inserted first, followed by cast1 during the
visite, since we need do it in bottom-up ( when we do getBestMatch() for f1
and insert cast1, we need know the output type of cast2, in order to
determine output type of f2(), which is argument to f1() ).
Please let me know your thoughts. Thanks!
Re: Discussion of implicit cast design in drill engine.
Posted by Jinfeng Ni <ji...@gmail.com>.
I put the discussion under JIRA-259, which was created by Yash for the cast
functionality proposal.
On Sun, Dec 15, 2013 at 9:57 PM, Jacques Nadeau <ja...@apache.org> wrote:
> This looks good. Do you have a JIRA to attach this work/discussion?
>
> thx,
> j
>
>
> On Mon, Dec 9, 2013 at 10:16 PM, Jinfeng Ni <ji...@gmail.com> wrote:
>
> > Hi Yash / All,
> >
> > I gave some thoughts about how to implement implicit cast. Here is my
> > preliminary thoughts.
> >
> > 1 . implicit cast vs explicit cast.
> >
> > Basically, implicit cast would need leverage explicit cast's
> > implementation. For instance, given an logical expression : 1 + 3.0, if
> > the function resolver finds that argument 1 should be implicitly casted
> > into float4, then, drill code should transform the logical expression
> into
> > cast(1 as float4) + 3.0, so that "+" operator will call the add(float4,
> > float4) implementation.
> >
> > 2. Add implicit cast function call in the logical expression tree.
> >
> > Currently, drill code uses
> > FunctionImplementationRegistry.getFunction(FunctionCall)
> > to get DrillFuncHolder in EvalVisitor.
> >
> > Your FunctionResolver is used to find the best match in the call of
> > getFunction(). However, if the best match says implicit cast is required,
> > it's kind of difficult to let getFunction() do 1) insert the cast
> function
> > to the logical expression tree, and 2) get cast's DrillFuncHolder, and 3)
> > generate the code for the cast function.
> >
> > We probably should separate the process of adding implicit cast from the
> > logic of FunctionImplementationRegistry.getFunction() and code
> generation.
> >
> > To do that :
> >
> > - Introduce a new Visitor class ImplicitCastBuilder.
> > - ImplicitCastBuilder will look similar to EvalVisitor .
> > ImplicitCastBuilder extends AbstractExprVisitor<*LogicalExpression*,
> > CodeGenerator<?>, RuntimeException>
> > - ImplicitCastBuilder should build cast function call in
> *bottom-up*
> > way.
> > - ImplicitCastBuilder will modify logical expression tree, and
> > replace an argument with a cast function call, if your
> > *FunctionResolver.getBestMatch() *shows implicit cast is required.
> > - CodeGenerator.addExpr will call ImplicitCastBuilder to insert the
> > implicit cast() to logical expression tree .
> >
> >
> > public HoldingContainer addExpr(LogicalExpression ex, boolean rotate){
> >
> > // logger.debug("Adding next write {}", ex);
> >
> > if(rotate) rotateBlock();
> >
> > *ex = implicitCastBuilder.accept(ex, this);*
> >
> > return evaluationVisitor.addExpr(ex, this);
> >
> > }
> >
> >
> > - EvalVisitor will then do the match() and code generation as
> before.
> > ( No need to call your *FunctionResolver.getBestMatch() *at this
> stage,
> > since all the required implicit cast has been inserted into the
> logical
> > expression tree).
> >
> >
> > For example, let's say we have logical expression tree f1 ( a1, f2( a2,
> > a3))
> >
> > f1()
> > / \
> > / \
> > a1 f2()
> > / \
> > / \
> > a2 a3
> >
> > We may end up with the following logical expression tree, after
> > ImplicitCastBuilder visit.
> >
> > f1()
> > / \
> > / \
> > a1 cast1()
> > \
> > \
> > f2()
> > / \
> > / \
> > a2 cast2()
> > |
> > |
> > a3
> >
> > Note that cast2 would be inserted first, followed by cast1 during the
> > visite, since we need do it in bottom-up ( when we do getBestMatch() for
> f1
> > and insert cast1, we need know the output type of cast2, in order to
> > determine output type of f2(), which is argument to f1() ).
> >
> > Please let me know your thoughts. Thanks!
> >
>
Re: Discussion of implicit cast design in drill engine.
Posted by Jacques Nadeau <ja...@apache.org>.
This looks good. Do you have a JIRA to attach this work/discussion?
thx,
j
On Mon, Dec 9, 2013 at 10:16 PM, Jinfeng Ni <ji...@gmail.com> wrote:
> Hi Yash / All,
>
> I gave some thoughts about how to implement implicit cast. Here is my
> preliminary thoughts.
>
> 1 . implicit cast vs explicit cast.
>
> Basically, implicit cast would need leverage explicit cast's
> implementation. For instance, given an logical expression : 1 + 3.0, if
> the function resolver finds that argument 1 should be implicitly casted
> into float4, then, drill code should transform the logical expression into
> cast(1 as float4) + 3.0, so that "+" operator will call the add(float4,
> float4) implementation.
>
> 2. Add implicit cast function call in the logical expression tree.
>
> Currently, drill code uses
> FunctionImplementationRegistry.getFunction(FunctionCall)
> to get DrillFuncHolder in EvalVisitor.
>
> Your FunctionResolver is used to find the best match in the call of
> getFunction(). However, if the best match says implicit cast is required,
> it's kind of difficult to let getFunction() do 1) insert the cast function
> to the logical expression tree, and 2) get cast's DrillFuncHolder, and 3)
> generate the code for the cast function.
>
> We probably should separate the process of adding implicit cast from the
> logic of FunctionImplementationRegistry.getFunction() and code generation.
>
> To do that :
>
> - Introduce a new Visitor class ImplicitCastBuilder.
> - ImplicitCastBuilder will look similar to EvalVisitor .
> ImplicitCastBuilder extends AbstractExprVisitor<*LogicalExpression*,
> CodeGenerator<?>, RuntimeException>
> - ImplicitCastBuilder should build cast function call in *bottom-up*
> way.
> - ImplicitCastBuilder will modify logical expression tree, and
> replace an argument with a cast function call, if your
> *FunctionResolver.getBestMatch() *shows implicit cast is required.
> - CodeGenerator.addExpr will call ImplicitCastBuilder to insert the
> implicit cast() to logical expression tree .
>
>
> public HoldingContainer addExpr(LogicalExpression ex, boolean rotate){
>
> // logger.debug("Adding next write {}", ex);
>
> if(rotate) rotateBlock();
>
> *ex = implicitCastBuilder.accept(ex, this);*
>
> return evaluationVisitor.addExpr(ex, this);
>
> }
>
>
> - EvalVisitor will then do the match() and code generation as before.
> ( No need to call your *FunctionResolver.getBestMatch() *at this stage,
> since all the required implicit cast has been inserted into the logical
> expression tree).
>
>
> For example, let's say we have logical expression tree f1 ( a1, f2( a2,
> a3))
>
> f1()
> / \
> / \
> a1 f2()
> / \
> / \
> a2 a3
>
> We may end up with the following logical expression tree, after
> ImplicitCastBuilder visit.
>
> f1()
> / \
> / \
> a1 cast1()
> \
> \
> f2()
> / \
> / \
> a2 cast2()
> |
> |
> a3
>
> Note that cast2 would be inserted first, followed by cast1 during the
> visite, since we need do it in bottom-up ( when we do getBestMatch() for f1
> and insert cast1, we need know the output type of cast2, in order to
> determine output type of f2(), which is argument to f1() ).
>
> Please let me know your thoughts. Thanks!
>