Working with a RexCall tree in Filter

["Hartman, Trevor" <th...@ebay.com>]

I'm working on pushing down filtering. In my own Filter rel impl, I see `condition` is represented as a RexCall, with the appropriate operations and operands.

e.g. where id = 1 AND (photoCount > 0 OR quantity > 0) builds up a RexCall tree with AND as the root op.

To implement filtering, I need a method like:
def passesFilter(expr: RexCall, fields: Map[String, Any]): Boolean

As I scan my table, I'd check passesFilter for each row, passing in a Map with keys id, photoCount and quantity in this case.

In reality, I'd probably use a translator (implements RexVisitorImpl) to turn the RexCall into a data structure that's easier to work with in Scala. However, looking through Calcite source, I see some things like RexExecutor, which led me to wonder:

Is there anything in Calcite that can help me define `passesFilter` without me having to manually walk an expression tree and evaluate the conditions? The built-in Filter implementation must be doing this already, but I'm not sure how or if I can reuse that.

Thanks,
Trevor

Re: Working with a RexCall tree in Filter

[Milinda Pathirage <mp...@umail.iu.edu>]

Hi Trevor,

You should probably have a look at the
*org.apache.calcite.interpreter.JaninoRexCompiler*. This is  used
inside *org.apache.calcite.interpreter.FilterNode.
*As I understand this RexCompiler converts the condition into a
implementation of org.apache.calcite.interpreter.Scalar which can be used
to evaluate the condition against the input values.

Thanks
Milinda

On Fri, Mar 6, 2015 at 1:02 PM, Hartman, Trevor <th...@ebay.com> wrote:

> I'm working on pushing down filtering. In my own Filter rel impl, I see
> `condition` is represented as a RexCall, with the appropriate operations
> and operands.
>
> e.g. where id = 1 AND (photoCount > 0 OR quantity > 0) builds up a RexCall
> tree with AND as the root op.
>
> To implement filtering, I need a method like:
> def passesFilter(expr: RexCall, fields: Map[String, Any]): Boolean
>
> As I scan my table, I'd check passesFilter for each row, passing in a Map
> with keys id, photoCount and quantity in this case.
>
> In reality, I'd probably use a translator (implements RexVisitorImpl) to
> turn the RexCall into a data structure that's easier to work with in Scala.
> However, looking through Calcite source, I see some things like
> RexExecutor, which led me to wonder:
>
> Is there anything in Calcite that can help me define `passesFilter`
> without me having to manually walk an expression tree and evaluate the
> conditions? The built-in Filter implementation must be doing this already,
> but I'm not sure how or if I can reuse that.
>
> Thanks,
> Trevor
>



-- 
Milinda Pathirage

PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University

twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org

Re: Working with a RexCall tree in Filter

["Hartman, Trevor" <th...@ebay.com>]

Thanks, working through these answers...

Trevor

Re: Working with a RexCall tree in Filter

[Julian Hyde <ju...@hydromatic.net>]

There are two classical approaches to expression evaluation - compilation and interpreting. Calcite has both:

EnumerableFilter uses compilation. It converts its expression into Java code, using a RexToLixTranslator (more precisely the EnumerableFilter is converted first to an EnumerableCalc, which is capable of doing filters and projects; see how EnumerableCalc.implement calls RexToLixTranslator.convert).

BindableFilter uses interpretation. It creates a FilterNode, which calls Interpreter.compile (a slight misnomer), which creates a Scalar. You can call scalar.execute(Context) and convert the result to a Boolean.

(The current implementation of Interpreter.compile happens to work by invoking the code-generation but there’s no reason why it should remain that way.)

Julian


On Mar 6, 2015, at 10:02 AM, Hartman, Trevor <th...@ebay.com> wrote:

> I'm working on pushing down filtering. In my own Filter rel impl, I see `condition` is represented as a RexCall, with the appropriate operations and operands.
> 
> e.g. where id = 1 AND (photoCount > 0 OR quantity > 0) builds up a RexCall tree with AND as the root op.
> 
> To implement filtering, I need a method like:
> def passesFilter(expr: RexCall, fields: Map[String, Any]): Boolean
> 
> As I scan my table, I'd check passesFilter for each row, passing in a Map with keys id, photoCount and quantity in this case.
> 
> In reality, I'd probably use a translator (implements RexVisitorImpl) to turn the RexCall into a data structure that's easier to work with in Scala. However, looking through Calcite source, I see some things like RexExecutor, which led me to wonder:
> 
> Is there anything in Calcite that can help me define `passesFilter` without me having to manually walk an expression tree and evaluate the conditions? The built-in Filter implementation must be doing this already, but I'm not sure how or if I can reuse that.
> 
> Thanks,
> Trevor