You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Pranav Deshpande <de...@gmail.com> on 2022/09/23 20:50:25 UTC
Conversion from Bindable to Enumerable
Dear Apache Calcite Team,
Is there a way to convert a BindableRel into an EnumerableRel?
There is a rule to convert an EnumerableRel into a BindableRel, but not the
other way around.
I am asking this because I wish to have my TableScans use the
BindableConvention and the rest of my execution using the Enumerable
Convention [as it has more operators!].
Regards,
Pranav
Re: Conversion from Bindable to Enumerable
Posted by Pranav Deshpande <de...@gmail.com>.
Thank you very much Thomas & Stamatis.
Thomas, I will try what you said.
Stamatis, I looked at the rule and the test case. And yes, it is exactly
what I wanted to try! Thank you very much.
I am facing one issue though. I have a test case where I have all the
enumerable rules. It is working. When I remove the enumerable table scan
and add the bindable table scan, it gives me the following error upon
execution:
*NullPointerException: at HepPlanner.addRelToGraph(...)*
*.....*
*at Interpreter.optimize(..)*
*....*
*Baz.bind (unknown source).*
My planner is constructed in a manner similar to the one created by you in
this tutorial. https://www.youtube.com/watch?v=p1O3E33FIs8
Is this because I am missing something (like some rules)? Where can I view
the planner constructed by the CalciteAssert class?
Thanks & Regards,.
Pranav
On Tue, Sep 27, 2022 at 2:01 PM Stamatis Zampetakis <za...@gmail.com>
wrote:
> Hi Pranav,
>
> The rule that you are probably looking for is EnumerableInterpreterRule
> [1].
>
> The ScannableTableTest [2] contains a bunch of test cases very similar to
> what you want to achieve. If you want to see how everything works together,
> pick one test and run it in debug mode.
>
> Best,
> Stamatis
>
> [1]
>
> https://github.com/apache/calcite/blob/a505b25eacc473c6ec0ef8abd40c1ccae86297b6/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpreterRule.java
> [2]
>
> https://github.com/apache/calcite/blob/a505b25eacc473c6ec0ef8abd40c1ccae86297b6/core/src/test/java/org/apache/calcite/test/ScannableTableTest.java
>
> On Tue, Sep 27, 2022 at 12:20 PM Thomas Rebele <tr...@tibco.com.invalid>
> wrote:
>
> > Hello Pranav,
> >
> > here some untested pseudo-code that could give you an idea how to achieve
> > this. It probably needs several adaptations to make it work (marked with
> > "<- here", possibly others).
> >
> > private static final Method BIND = Types.lookupMethod(Bindable.class,
> > > "bind", Bindable.class, DataContext.class); // <- here
> > >
> > > private static class EnumerableBindableRel extends AbstractRelNode
> > > implements EnumerableRel {
> > > private final BindableRel bindable;
> > >
> > > public EnumerableBindableRel(BindableRel bindable) {
> > > super(bindable.getCluster(), bindable.getTraitSet());
> > > this.bindable = bindable;
> > > }
> > >
> > > @Override
> > > public Result implement(EnumerableRelImplementor implementor,
> Prefer
> > > pref) {
> > > final PhysType physType = PhysTypeImpl.of(
> > > implementor.getTypeFactory(),
> > > this.getRowType(),
> > > JavaRowFormat.ARRAY); // <- here
> > >
> > > final BlockBuilder builder = new BlockBuilder();
> > > MethodCallExpression call = Expressions.call( // <- here,
> correct
> > > arguments?
> > > implementor.stash(this.bindable, Bindable.class),
> > > BIND,
> > > implementor.getRootExpression());
> > >
> > > ParameterExpression resultVar =
> > > Expressions.parameter(Enumerable.class, "_result");
> > > builder.add(Expressions.declare(Modifier.FINAL, resultVar,
> > call));
> > > builder.add(Expressions.return_(null, resultVar));
> > > return implementor.result(physType, builder.toBlock());
> > > }
> > > }
> > >
> > > private static EnumerableRel getEnumerableRel(BindableRel bindable) {
> > > return new EnumerableBindableRel(bindable);
> > > }
> > >
> >
> > Cordialement / Best Regards,
> > Dr. Thomas Rebele | R&D Developer | Germany | E trebele@tibco.com | W
> > www.tibco.com
> >
> > TIBCO Software GmbH | St.-Martin-Str. 106, 81669 München, Deutschland |
> > Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer:
> William
> > Hughes; Alexander E. Kolar
> >
> > Cordialement / Best Regards,
> > *Dr. Thomas Rebele* | R&D Developer | Germany | *E* *trebele@tibco.com
> > <tr...@tibco.com>* | *W* *www.tibco.com <http://www.tibco.com/>*
> >
> > *TIBCO Software GmbH* | St.-Martin-Str. 106, 81669 München, Deutschland
> |
> > Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer:
> William
> > Hughes; Alexander E. Kolar
> >
> >
> >
> > On Fri, Sep 23, 2022 at 10:50 PM Pranav Deshpande <
> > deshpande.v.pranav@gmail.com> wrote:
> >
> > > Dear Apache Calcite Team,
> > > Is there a way to convert a BindableRel into an EnumerableRel?
> > >
> > > There is a rule to convert an EnumerableRel into a BindableRel, but not
> > the
> > > other way around.
> > >
> > > I am asking this because I wish to have my TableScans use the
> > > BindableConvention and the rest of my execution using the Enumerable
> > > Convention [as it has more operators!].
> > >
> > > Regards,
> > > Pranav
> > >
> >
>
Re: Conversion from Bindable to Enumerable
Posted by Stamatis Zampetakis <za...@gmail.com>.
Hi Pranav,
The rule that you are probably looking for is EnumerableInterpreterRule [1].
The ScannableTableTest [2] contains a bunch of test cases very similar to
what you want to achieve. If you want to see how everything works together,
pick one test and run it in debug mode.
Best,
Stamatis
[1]
https://github.com/apache/calcite/blob/a505b25eacc473c6ec0ef8abd40c1ccae86297b6/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableInterpreterRule.java
[2]
https://github.com/apache/calcite/blob/a505b25eacc473c6ec0ef8abd40c1ccae86297b6/core/src/test/java/org/apache/calcite/test/ScannableTableTest.java
On Tue, Sep 27, 2022 at 12:20 PM Thomas Rebele <tr...@tibco.com.invalid>
wrote:
> Hello Pranav,
>
> here some untested pseudo-code that could give you an idea how to achieve
> this. It probably needs several adaptations to make it work (marked with
> "<- here", possibly others).
>
> private static final Method BIND = Types.lookupMethod(Bindable.class,
> > "bind", Bindable.class, DataContext.class); // <- here
> >
> > private static class EnumerableBindableRel extends AbstractRelNode
> > implements EnumerableRel {
> > private final BindableRel bindable;
> >
> > public EnumerableBindableRel(BindableRel bindable) {
> > super(bindable.getCluster(), bindable.getTraitSet());
> > this.bindable = bindable;
> > }
> >
> > @Override
> > public Result implement(EnumerableRelImplementor implementor, Prefer
> > pref) {
> > final PhysType physType = PhysTypeImpl.of(
> > implementor.getTypeFactory(),
> > this.getRowType(),
> > JavaRowFormat.ARRAY); // <- here
> >
> > final BlockBuilder builder = new BlockBuilder();
> > MethodCallExpression call = Expressions.call( // <- here, correct
> > arguments?
> > implementor.stash(this.bindable, Bindable.class),
> > BIND,
> > implementor.getRootExpression());
> >
> > ParameterExpression resultVar =
> > Expressions.parameter(Enumerable.class, "_result");
> > builder.add(Expressions.declare(Modifier.FINAL, resultVar,
> call));
> > builder.add(Expressions.return_(null, resultVar));
> > return implementor.result(physType, builder.toBlock());
> > }
> > }
> >
> > private static EnumerableRel getEnumerableRel(BindableRel bindable) {
> > return new EnumerableBindableRel(bindable);
> > }
> >
>
> Cordialement / Best Regards,
> Dr. Thomas Rebele | R&D Developer | Germany | E trebele@tibco.com | W
> www.tibco.com
>
> TIBCO Software GmbH | St.-Martin-Str. 106, 81669 München, Deutschland |
> Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer: William
> Hughes; Alexander E. Kolar
>
> Cordialement / Best Regards,
> *Dr. Thomas Rebele* | R&D Developer | Germany | *E* *trebele@tibco.com
> <tr...@tibco.com>* | *W* *www.tibco.com <http://www.tibco.com/>*
>
> *TIBCO Software GmbH* | St.-Martin-Str. 106, 81669 München, Deutschland |
> Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer: William
> Hughes; Alexander E. Kolar
>
>
>
> On Fri, Sep 23, 2022 at 10:50 PM Pranav Deshpande <
> deshpande.v.pranav@gmail.com> wrote:
>
> > Dear Apache Calcite Team,
> > Is there a way to convert a BindableRel into an EnumerableRel?
> >
> > There is a rule to convert an EnumerableRel into a BindableRel, but not
> the
> > other way around.
> >
> > I am asking this because I wish to have my TableScans use the
> > BindableConvention and the rest of my execution using the Enumerable
> > Convention [as it has more operators!].
> >
> > Regards,
> > Pranav
> >
>
Re: Conversion from Bindable to Enumerable
Posted by Thomas Rebele <tr...@tibco.com.INVALID>.
Hello Pranav,
here some untested pseudo-code that could give you an idea how to achieve
this. It probably needs several adaptations to make it work (marked with
"<- here", possibly others).
private static final Method BIND = Types.lookupMethod(Bindable.class,
> "bind", Bindable.class, DataContext.class); // <- here
>
> private static class EnumerableBindableRel extends AbstractRelNode
> implements EnumerableRel {
> private final BindableRel bindable;
>
> public EnumerableBindableRel(BindableRel bindable) {
> super(bindable.getCluster(), bindable.getTraitSet());
> this.bindable = bindable;
> }
>
> @Override
> public Result implement(EnumerableRelImplementor implementor, Prefer
> pref) {
> final PhysType physType = PhysTypeImpl.of(
> implementor.getTypeFactory(),
> this.getRowType(),
> JavaRowFormat.ARRAY); // <- here
>
> final BlockBuilder builder = new BlockBuilder();
> MethodCallExpression call = Expressions.call( // <- here, correct
> arguments?
> implementor.stash(this.bindable, Bindable.class),
> BIND,
> implementor.getRootExpression());
>
> ParameterExpression resultVar =
> Expressions.parameter(Enumerable.class, "_result");
> builder.add(Expressions.declare(Modifier.FINAL, resultVar, call));
> builder.add(Expressions.return_(null, resultVar));
> return implementor.result(physType, builder.toBlock());
> }
> }
>
> private static EnumerableRel getEnumerableRel(BindableRel bindable) {
> return new EnumerableBindableRel(bindable);
> }
>
Cordialement / Best Regards,
Dr. Thomas Rebele | R&D Developer | Germany | E trebele@tibco.com | W
www.tibco.com
TIBCO Software GmbH | St.-Martin-Str. 106, 81669 München, Deutschland |
Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer: William
Hughes; Alexander E. Kolar
Cordialement / Best Regards,
*Dr. Thomas Rebele* | R&D Developer | Germany | *E* *trebele@tibco.com
<tr...@tibco.com>* | *W* *www.tibco.com <http://www.tibco.com/>*
*TIBCO Software GmbH* | St.-Martin-Str. 106, 81669 München, Deutschland |
Registergericht: Amtsgericht München, HRB 123355 | Geschäftsführer: William
Hughes; Alexander E. Kolar
On Fri, Sep 23, 2022 at 10:50 PM Pranav Deshpande <
deshpande.v.pranav@gmail.com> wrote:
> Dear Apache Calcite Team,
> Is there a way to convert a BindableRel into an EnumerableRel?
>
> There is a rule to convert an EnumerableRel into a BindableRel, but not the
> other way around.
>
> I am asking this because I wish to have my TableScans use the
> BindableConvention and the rest of my execution using the Enumerable
> Convention [as it has more operators!].
>
> Regards,
> Pranav
>