You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Bhavya Aggarwal <bh...@knoldus.com> on 2020/12/13 04:41:51 UTC
Need a Jason output if physical tree
Hi All,
We need to generate a JSON object for the physical execution tree that has
been created. Is there an option in Calcite that we can use to do this. I
am not sure what is the right approach to do it. Please let me know if
there are different ways to achieve this.
Regards
Bhavya
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Vladmir,
Thanks for your feedback.
IMO, Root[$0] should depend on t.a and t.b, because the value of t.b
decides if some value could be included in Root[$0].
However, after investigating the code, I find you are right!
The current implementation of RelMdColumnOrigins#getColumnOrigins(Filter,
RelMetadataQuery, int) does not incorporate columns referenced in the
condition.
I am wondering if this is by design? Do we need another metadata query to
evaluate the relation between input and output columns?
Best,
Liya Fan
On Tue, Dec 22, 2020 at 3:50 PM Vladimir Ozerov <pp...@gmail.com> wrote:
> Hi Liya,
>
> This will not work AFAIK. Consider the query "SELECT a FROM t WHERE b>1".
> The top-level operator has only one column:
>
> Root[$0]
> Project[$0]
> Filter[$1>1]
> Scan[$0=a, $1=b]
>
> If you invoke RelMdColumnOrigins on Root[$0], you will get [t.a], but miss
> [t.b].
> To my knowledge, rules are the only way to reliably. constrain columns
> returned from the scan.
>
> Regards,
> Vladimir.
>
> вт, 22 дек. 2020 г. в 05:14, Fan Liya <li...@gmail.com>:
>
> > Hi Bhavya,
> >
> > Sorry I am not sure if I fully understand your question.
> >
> > Let me try to answer it according to my understanding:
> >
> > 1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object,
> which
> > includes a RelOptTable object.
> > 2. The table scan also has a RelOptTable object, and all table scans of
> the
> > plan can be found (e.g. through a visitor)
> > 3. With the information of 1 and 2, given any output column, we can get
> to
> > know it is derived from which columns from which table scans.
> > 4. With the information of 3, given a table scan, we can get to know
> which
> > column is never used in any output columns, and such columns can be
> pruned.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Hi Liya,
> > >
> > > I had a look at the RelColumnMDOrigins and it is useful in determining
> > > which columns are from which table but still I am not sure how can I
> get
> > > the column information for TableScan without the rules. If you have any
> > > specific example where we have used this approach will be really
> helpful
> > to
> > > me.
> > >
> > > Thanks and regards
> > > Bhavya
> > >
> > > On Mon, Dec 21, 2020 at 5:53 PM Fan Liya <li...@gmail.com> wrote:
> > >
> > > > Hi Bhavya,
> > > >
> > > > IMO, to solve the problem from a global view, the following steps
> needs
> > > to
> > > > be taken:
> > > >
> > > > 1. Generate a physical plan in the original way (without considering
> > > column
> > > > pruning in the table scan)
> > > > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > > > utility (the details have been described above)
> > > > 3. Post process the plan with one of the following ways:
> > > > a) a plan visitor that adjusts other operators in the tree.
> > > > b) a light-weight planner (Hep or Volcano with limited rule sets)
> > > >
> > > > Run the query with the finally generated plan.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > > wrote:
> > > >
> > > > > Hi Fan,
> > > > >
> > > > > I looked at the class RelColumnMDOrigins and it is giving me the
> > origin
> > > > of
> > > > > the column, but even if I want to take it as a global decision I am
> > not
> > > > > sure how to proceed. Can you please elaborate on how to achieve
> this
> > ?
> > > I
> > > > am
> > > > > literally stuck as I do not want to use so many rules as in any
> case
> > I
> > > > have
> > > > > to pass these to the TableScan, even if the user does a select *
> from
> > > > > table, I need to add all those columns to the table scan.
> > > > >
> > > > > Regards
> > > > > Bhavya
> > > > >
> > > > > --
> > > > > Your feedback matters - At Knoldus we aim to be very professional
> in
> > > our
> > > > > quality of work, commitment to results, and proactive
> communication.
> > If
> > > > > you
> > > > > feel otherwise please share your feedback
> > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
Re: Need a Jason output if physical tree
Posted by Vladimir Ozerov <pp...@gmail.com>.
Hi Liya,
This will not work AFAIK. Consider the query "SELECT a FROM t WHERE b>1".
The top-level operator has only one column:
Root[$0]
Project[$0]
Filter[$1>1]
Scan[$0=a, $1=b]
If you invoke RelMdColumnOrigins on Root[$0], you will get [t.a], but miss
[t.b].
To my knowledge, rules are the only way to reliably. constrain columns
returned from the scan.
Regards,
Vladimir.
вт, 22 дек. 2020 г. в 05:14, Fan Liya <li...@gmail.com>:
> Hi Bhavya,
>
> Sorry I am not sure if I fully understand your question.
>
> Let me try to answer it according to my understanding:
>
> 1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object, which
> includes a RelOptTable object.
> 2. The table scan also has a RelOptTable object, and all table scans of the
> plan can be found (e.g. through a visitor)
> 3. With the information of 1 and 2, given any output column, we can get to
> know it is derived from which columns from which table scans.
> 4. With the information of 3, given a table scan, we can get to know which
> column is never used in any output columns, and such columns can be pruned.
>
> Best,
> Liya Fan
>
>
> On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Hi Liya,
> >
> > I had a look at the RelColumnMDOrigins and it is useful in determining
> > which columns are from which table but still I am not sure how can I get
> > the column information for TableScan without the rules. If you have any
> > specific example where we have used this approach will be really helpful
> to
> > me.
> >
> > Thanks and regards
> > Bhavya
> >
> > On Mon, Dec 21, 2020 at 5:53 PM Fan Liya <li...@gmail.com> wrote:
> >
> > > Hi Bhavya,
> > >
> > > IMO, to solve the problem from a global view, the following steps needs
> > to
> > > be taken:
> > >
> > > 1. Generate a physical plan in the original way (without considering
> > column
> > > pruning in the table scan)
> > > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > > utility (the details have been described above)
> > > 3. Post process the plan with one of the following ways:
> > > a) a plan visitor that adjusts other operators in the tree.
> > > b) a light-weight planner (Hep or Volcano with limited rule sets)
> > >
> > > Run the query with the finally generated plan.
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > wrote:
> > >
> > > > Hi Fan,
> > > >
> > > > I looked at the class RelColumnMDOrigins and it is giving me the
> origin
> > > of
> > > > the column, but even if I want to take it as a global decision I am
> not
> > > > sure how to proceed. Can you please elaborate on how to achieve this
> ?
> > I
> > > am
> > > > literally stuck as I do not want to use so many rules as in any case
> I
> > > have
> > > > to pass these to the TableScan, even if the user does a select * from
> > > > table, I need to add all those columns to the table scan.
> > > >
> > > > Regards
> > > > Bhavya
> > > >
> > > > --
> > > > Your feedback matters - At Knoldus we aim to be very professional in
> > our
> > > > quality of work, commitment to results, and proactive communication.
> If
> > > > you
> > > > feel otherwise please share your feedback
> > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > >
> > >
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. <http://www.knoldus.com/>
> > +91-9910483067
> > Canada - USA - India - Singapore
> > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak
> >
> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
>
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Bhavya,
Sorry I am not sure if I fully understand your question.
Let me try to answer it according to my understanding:
1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object, which
includes a RelOptTable object.
2. The table scan also has a RelOptTable object, and all table scans of the
plan can be found (e.g. through a visitor)
3. With the information of 1 and 2, given any output column, we can get to
know it is derived from which columns from which table scans.
4. With the information of 3, given a table scan, we can get to know which
column is never used in any output columns, and such columns can be pruned.
Best,
Liya Fan
On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Hi Liya,
>
> I had a look at the RelColumnMDOrigins and it is useful in determining
> which columns are from which table but still I am not sure how can I get
> the column information for TableScan without the rules. If you have any
> specific example where we have used this approach will be really helpful to
> me.
>
> Thanks and regards
> Bhavya
>
> On Mon, Dec 21, 2020 at 5:53 PM Fan Liya <li...@gmail.com> wrote:
>
> > Hi Bhavya,
> >
> > IMO, to solve the problem from a global view, the following steps needs
> to
> > be taken:
> >
> > 1. Generate a physical plan in the original way (without considering
> column
> > pruning in the table scan)
> > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > utility (the details have been described above)
> > 3. Post process the plan with one of the following ways:
> > a) a plan visitor that adjusts other operators in the tree.
> > b) a light-weight planner (Hep or Volcano with limited rule sets)
> >
> > Run the query with the finally generated plan.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Hi Fan,
> > >
> > > I looked at the class RelColumnMDOrigins and it is giving me the origin
> > of
> > > the column, but even if I want to take it as a global decision I am not
> > > sure how to proceed. Can you please elaborate on how to achieve this ?
> I
> > am
> > > literally stuck as I do not want to use so many rules as in any case I
> > have
> > > to pass these to the TableScan, even if the user does a select * from
> > > table, I need to add all those columns to the table scan.
> > >
> > > Regards
> > > Bhavya
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Hi Liya,
I had a look at the RelColumnMDOrigins and it is useful in determining
which columns are from which table but still I am not sure how can I get
the column information for TableScan without the rules. If you have any
specific example where we have used this approach will be really helpful to
me.
Thanks and regards
Bhavya
On Mon, Dec 21, 2020 at 5:53 PM Fan Liya <li...@gmail.com> wrote:
> Hi Bhavya,
>
> IMO, to solve the problem from a global view, the following steps needs to
> be taken:
>
> 1. Generate a physical plan in the original way (without considering column
> pruning in the table scan)
> 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> utility (the details have been described above)
> 3. Post process the plan with one of the following ways:
> a) a plan visitor that adjusts other operators in the tree.
> b) a light-weight planner (Hep or Volcano with limited rule sets)
>
> Run the query with the finally generated plan.
>
> Best,
> Liya Fan
>
>
> On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Hi Fan,
> >
> > I looked at the class RelColumnMDOrigins and it is giving me the origin
> of
> > the column, but even if I want to take it as a global decision I am not
> > sure how to proceed. Can you please elaborate on how to achieve this ? I
> am
> > literally stuck as I do not want to use so many rules as in any case I
> have
> > to pass these to the TableScan, even if the user does a select * from
> > table, I need to add all those columns to the table scan.
> >
> > Regards
> > Bhavya
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
>
--
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. <http://www.knoldus.com/>
+91-9910483067
Canada - USA - India - Singapore
<https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
<https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Bhavya,
IMO, to solve the problem from a global view, the following steps needs to
be taken:
1. Generate a physical plan in the original way (without considering column
pruning in the table scan)
2. Modify all the table scans in the plan with the RelColumnMDOrigins
utility (the details have been described above)
3. Post process the plan with one of the following ways:
a) a plan visitor that adjusts other operators in the tree.
b) a light-weight planner (Hep or Volcano with limited rule sets)
Run the query with the finally generated plan.
Best,
Liya Fan
On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Hi Fan,
>
> I looked at the class RelColumnMDOrigins and it is giving me the origin of
> the column, but even if I want to take it as a global decision I am not
> sure how to proceed. Can you please elaborate on how to achieve this ? I am
> literally stuck as I do not want to use so many rules as in any case I have
> to pass these to the TableScan, even if the user does a select * from
> table, I need to add all those columns to the table scan.
>
> Regards
> Bhavya
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Hi Fan,
I looked at the class RelColumnMDOrigins and it is giving me the origin of
the column, but even if I want to take it as a global decision I am not
sure how to proceed. Can you please elaborate on how to achieve this ? I am
literally stuck as I do not want to use so many rules as in any case I have
to pass these to the TableScan, even if the user does a select * from
table, I need to add all those columns to the table scan.
Regards
Bhavya
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Vladmir,
Thanks for your feedback.
I see your point. The example I gave above was not a good one.
I agree with you that making the decision locally solves most problems.
However, it may be sub-optimal for some scenarios:
The project operator may contain unnecessary columns (even if the columns
appear in the query). For example, the optimizer may determine that the
column is always a constant, and should only be included in the final write.
Best,
Liya Fan
On Fri, Dec 18, 2020 at 9:32 PM Vladimir Ozerov <pp...@gmail.com> wrote:
> Hi Liya,
>
> I do not think, this is a global decision. If you go bottom-up in the plan,
> and find the Project operator, than none parent operators will use any
> columns from the lower scans, other than those present in the Project.
>
> It is a common practice for Calcite integrations to restrict the number of
> columns returned from scans. It is achieved as follows:
> 1) There should be rules to extract projections from some operators (e.g.
> Joins)
> 2) There should be rules to transpose projects with child operators
> 3) Finally, you should create a rule to move project into scan for the
> pattern [Project<-Scan]
>
> When all three prerequisites are ready, the optimizer will produce missing
> projects (if any), then push them down the operator tree, and finally merge
> into the scan.
>
> Regards,
> Vladimir
>
> Пт, 18 дек. 2020 г. в 13:34, Fan Liya <li...@gmail.com>:
>
> > Hi Bhavya,
> >
> > Thanks for your explanation.
> >
> > I do not think a rule like ProjectFilterTableScan solves the problem.
> > To see this, please note that the parent operator of the LogicalProject
> may
> > need other columns from the table
> > (e.g. the parent operator may be a join, which uses some other columns as
> > the join key).
> >
> > So the column pruning of table scan should be a global decision,
> depending
> > on the whole plan.
> >
> > Calcite has RelMdColumnOrigins, which can help in this scenario.
> > In particular, given an output column in the output operator, it gives
> the
> > dependent columns in the underlying source tables.
> > So columns not referenced from the final output can be pruned from the
> > source table.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Hi Liya,
> > >
> > > Please see below , I have a query as given below
> > >
> > > SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
> > >
> > > Now I know that I have to fetch two columns from my columnar database
> > i.e.
> > > ID, Name, so need to pass these parameters in my custom Table Scan.
> > > But when I convert the LogicalTableScan to my custom Table scan I do
> not
> > > have that information, please see the tree below, in the Logical Table
> > Scan
> > > the Input is null, I was hoping that I will have the columns in the
> > > LogicalTableScan, The question is what do I need to do in Calcite to
> pass
> > > that information to the LogicalTableScan or in my physical operator.
> The
> > > way I have found is that I create a Rule like ProjectFilterTableScan
> rule
> > > and then extract what information I need. Is there a better way to do
> > this?
> > >
> > > -----------------Logical JSON Plan --------------------
> > > > {
> > > > "rels": [
> > > > {
> > > > "id": "0",
> > > > "relOp": "LogicalTableScan",
> > > > "table": [
> > > > "USERS"
> > > > ],
> > > > "inputs": []
> > > > },
> > > > {
> > > > "id": "1",
> > > > "relOp": "LogicalFilter",
> > > > "condition": {
> > > > "op": {
> > > > "name": "=",
> > > > "kind": "EQUALS",
> > > > "syntax": "BINARY"
> > > > },
> > > > "operands": [
> > > > {
> > > > "op": {
> > > > "name": "CAST",
> > > > "kind": "CAST",
> > > > "syntax": "SPECIAL"
> > > > },
> > > > "operands": [
> > > > {
> > > > "input": 1,
> > > > "name": "$1"
> > > > }
> > > > ],
> > > > "type": {
> > > > "type": "CHAR",
> > > > "nullable": true,
> > > > "precision": 5
> > > > }
> > > > },
> > > > {
> > > > "literal": "HELLO",
> > > > "type": {
> > > > "type": "CHAR",
> > > > "nullable": false,
> > > > "precision": 5
> > > > }
> > > > }
> > > > ]
> > > > }
> > > > },
> > > > {
> > > > "id": "2",
> > > > "relOp": "LogicalProject",
> > > > "fields": [
> > > > "ID"
> > > > ],
> > > > "exprs": [
> > > > {
> > > > "input": 0,
> > > > "name": "$0"
> > > > }
> > > > ]
> > > > }
> > > > ]
> > > > }
> > > >
> > > >> -----------------------------------------------------------
> > > >
> > > >
> > > Thanks and regards
> > > Bhavya
> > >
> > > On Fri, Dec 18, 2020 at 11:54 AM Fan Liya <li...@gmail.com>
> wrote:
> > >
> > > > Hi Bhavya,
> > > >
> > > > Sorry I do not understand your question. Why is it difficult to pass
> > sth.
> > > > to child operators?
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <bhavya@knoldus.com
> >
> > > > wrote:
> > > >
> > > > > Hi Liya,
> > > > >
> > > > > Actually the question is how to pass the projection columns to
> Table
> > > Scan
> > > > > as right now in the LogicalTableScan there are no projection
> columns
> > > > being
> > > > > passed so when I am trying to create my custom JSON , I do not have
> > > those
> > > > > projected columns or columns that are being used in the query. I
> want
> > > to
> > > > > understand what is the calcite mechanism for passing it to child
> > > > operators
> > > > > without using Rules as it will be a lot of rule that we need to
> > > > implement.
> > > > >
> > > > > Thanks and Regards
> > > > > Bhavya
> > > > >
> > > > > On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi Bhavya,
> > > > > >
> > > > > > It seems the problem is that the json format provided by Calcite
> > does
> > > > not
> > > > > > include properties specific to your operator?
> > > > > > If so, I think you can override RelNode#explainTerms in your
> > operator
> > > > to
> > > > > > attach the properties.
> > > > > >
> > > > > > Best,
> > > > > > Liya Fan
> > > > > >
> > > > > > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <
> > bhavya@knoldus.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Please let me know if we have an example for writing a custom
> > > visitor
> > > > > > that
> > > > > > > I can use. I see a RelVisitor class that I can extend but how
> to
> > > make
> > > > > it
> > > > > > to
> > > > > > > visit the specific Physical Operator.
> > > > > > >
> > > > > > > Thanks and Regards
> > > > > > > Bhavya
> > > > > > >
> > > > > > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <
> > > bhavya@knoldus.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > I tried using the RelJsonWriter and tried to add my custom
> > > > > > implementation
> > > > > > > > in the my custom Rel Nodes but I am not getting all the data
> in
> > > my
> > > > > > child
> > > > > > > > operators. We are trying to use Calcite for a columnar data
> and
> > > we
> > > > > need
> > > > > > > the
> > > > > > > > information of the projections in the TableScan as well
> because
> > > we
> > > > > need
> > > > > > > to
> > > > > > > > pass those projections to read specific columns but I am not
> > able
> > > > to
> > > > > > find
> > > > > > > > the projections in my Table scan. As per my understanding
> > Calcite
> > > > > > should
> > > > > > > be
> > > > > > > > passing this information as it is required for me reading the
> > > > table.
> > > > > > > Please
> > > > > > > > let me know if I need to do something special to get that
> > > > information
> > > > > > in
> > > > > > > > the child operator.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Bhavya
> > > > > > > >
> > > > > > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <
> > mmior@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >> That would likely be the best approach if you have some
> > specific
> > > > > JSON
> > > > > > > >> format you're trying to generate. If you're happy with a
> JSON
> > > > format
> > > > > > > >> defined by Calcite, you can look at RelJsonWriter.
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Michael Mior
> > > > > > > >> mmior@apache.org
> > > > > > > >>
> > > > > > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <
> > > > mgelbana@apache.org
> > > > > >
> > > > > > a
> > > > > > > >> écrit :
> > > > > > > >> >
> > > > > > > >> > I would use a visitor to traverse the optimized/physical
> > plan.
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> > > > > bhavya@knoldus.com
> > > > > > >
> > > > > > > >> wrote:
> > > > > > > >> >
> > > > > > > >> > > Hi All,
> > > > > > > >> > >
> > > > > > > >> > > We need to generate a JSON object for the physical
> > execution
> > > > > tree
> > > > > > > >> that has
> > > > > > > >> > > been created. Is there an option in Calcite that we can
> > use
> > > to
> > > > > do
> > > > > > > >> this. I
> > > > > > > >> > > am not sure what is the right approach to do it. Please
> > let
> > > me
> > > > > > know
> > > > > > > if
> > > > > > > >> > > there are different ways to achieve this.
> > > > > > > >> > >
> > > > > > > >> > > Regards
> > > > > > > >> > > Bhavya
> > > > > > > >> > >
> > > > > > > >> > > --
> > > > > > > >> > > Your feedback matters - At Knoldus we aim to be very
> > > > > professional
> > > > > > in
> > > > > > > >> our
> > > > > > > >> > > quality of work, commitment to results, and proactive
> > > > > > communication.
> > > > > > > >> If
> > > > > > > >> > > you
> > > > > > > >> > > feel otherwise please share your feedback
> > > > > > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work
> > on
> > > > it.
> > > > > > > >> > >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > *Bhavya Aggarwal*
> > > > > > > > CTO & Partner
> > > > > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > > > > +91-9910483067
> > > > > > > > Canada - USA - India - Singapore
> > > > > > > > <https://in.linkedin.com/company/knoldus> <
> > > > > > https://twitter.com/Knolspeak
> > > > > > > >
> > > > > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > > > > https://blog.knoldus.com/
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Bhavya Aggarwal*
> > > > > > > CTO & Partner
> > > > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > > > +91-9910483067
> > > > > > > Canada - USA - India - Singapore
> > > > > > > <https://in.linkedin.com/company/knoldus> <
> > > > > https://twitter.com/Knolspeak
> > > > > > >
> > > > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > > > https://blog.knoldus.com/
> > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Your feedback matters - At Knoldus we aim to be very
> professional
> > > in
> > > > > our
> > > > > > > quality of work, commitment to results, and proactive
> > > communication.
> > > > If
> > > > > > > you
> > > > > > > feel otherwise please share your feedback
> > > > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Bhavya Aggarwal*
> > > > > CTO & Partner
> > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > +91-9910483067
> > > > > Canada - USA - India - Singapore
> > > > > <https://in.linkedin.com/company/knoldus> <
> > > https://twitter.com/Knolspeak
> > > > >
> > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > https://blog.knoldus.com/
> > > >
> > > > >
> > > > > --
> > > > > Your feedback matters - At Knoldus we aim to be very professional
> in
> > > our
> > > > > quality of work, commitment to results, and proactive
> communication.
> > If
> > > > > you
> > > > > feel otherwise please share your feedback
> > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
Re: Need a Jason output if physical tree
Posted by Vladimir Ozerov <pp...@gmail.com>.
Hi Liya,
I do not think, this is a global decision. If you go bottom-up in the plan,
and find the Project operator, than none parent operators will use any
columns from the lower scans, other than those present in the Project.
It is a common practice for Calcite integrations to restrict the number of
columns returned from scans. It is achieved as follows:
1) There should be rules to extract projections from some operators (e.g.
Joins)
2) There should be rules to transpose projects with child operators
3) Finally, you should create a rule to move project into scan for the
pattern [Project<-Scan]
When all three prerequisites are ready, the optimizer will produce missing
projects (if any), then push them down the operator tree, and finally merge
into the scan.
Regards,
Vladimir
Пт, 18 дек. 2020 г. в 13:34, Fan Liya <li...@gmail.com>:
> Hi Bhavya,
>
> Thanks for your explanation.
>
> I do not think a rule like ProjectFilterTableScan solves the problem.
> To see this, please note that the parent operator of the LogicalProject may
> need other columns from the table
> (e.g. the parent operator may be a join, which uses some other columns as
> the join key).
>
> So the column pruning of table scan should be a global decision, depending
> on the whole plan.
>
> Calcite has RelMdColumnOrigins, which can help in this scenario.
> In particular, given an output column in the output operator, it gives the
> dependent columns in the underlying source tables.
> So columns not referenced from the final output can be pruned from the
> source table.
>
> Best,
> Liya Fan
>
>
> On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Hi Liya,
> >
> > Please see below , I have a query as given below
> >
> > SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
> >
> > Now I know that I have to fetch two columns from my columnar database
> i.e.
> > ID, Name, so need to pass these parameters in my custom Table Scan.
> > But when I convert the LogicalTableScan to my custom Table scan I do not
> > have that information, please see the tree below, in the Logical Table
> Scan
> > the Input is null, I was hoping that I will have the columns in the
> > LogicalTableScan, The question is what do I need to do in Calcite to pass
> > that information to the LogicalTableScan or in my physical operator. The
> > way I have found is that I create a Rule like ProjectFilterTableScan rule
> > and then extract what information I need. Is there a better way to do
> this?
> >
> > -----------------Logical JSON Plan --------------------
> > > {
> > > "rels": [
> > > {
> > > "id": "0",
> > > "relOp": "LogicalTableScan",
> > > "table": [
> > > "USERS"
> > > ],
> > > "inputs": []
> > > },
> > > {
> > > "id": "1",
> > > "relOp": "LogicalFilter",
> > > "condition": {
> > > "op": {
> > > "name": "=",
> > > "kind": "EQUALS",
> > > "syntax": "BINARY"
> > > },
> > > "operands": [
> > > {
> > > "op": {
> > > "name": "CAST",
> > > "kind": "CAST",
> > > "syntax": "SPECIAL"
> > > },
> > > "operands": [
> > > {
> > > "input": 1,
> > > "name": "$1"
> > > }
> > > ],
> > > "type": {
> > > "type": "CHAR",
> > > "nullable": true,
> > > "precision": 5
> > > }
> > > },
> > > {
> > > "literal": "HELLO",
> > > "type": {
> > > "type": "CHAR",
> > > "nullable": false,
> > > "precision": 5
> > > }
> > > }
> > > ]
> > > }
> > > },
> > > {
> > > "id": "2",
> > > "relOp": "LogicalProject",
> > > "fields": [
> > > "ID"
> > > ],
> > > "exprs": [
> > > {
> > > "input": 0,
> > > "name": "$0"
> > > }
> > > ]
> > > }
> > > ]
> > > }
> > >
> > >> -----------------------------------------------------------
> > >
> > >
> > Thanks and regards
> > Bhavya
> >
> > On Fri, Dec 18, 2020 at 11:54 AM Fan Liya <li...@gmail.com> wrote:
> >
> > > Hi Bhavya,
> > >
> > > Sorry I do not understand your question. Why is it difficult to pass
> sth.
> > > to child operators?
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > > On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > wrote:
> > >
> > > > Hi Liya,
> > > >
> > > > Actually the question is how to pass the projection columns to Table
> > Scan
> > > > as right now in the LogicalTableScan there are no projection columns
> > > being
> > > > passed so when I am trying to create my custom JSON , I do not have
> > those
> > > > projected columns or columns that are being used in the query. I want
> > to
> > > > understand what is the calcite mechanism for passing it to child
> > > operators
> > > > without using Rules as it will be a lot of rule that we need to
> > > implement.
> > > >
> > > > Thanks and Regards
> > > > Bhavya
> > > >
> > > > On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com>
> wrote:
> > > >
> > > > > Hi Bhavya,
> > > > >
> > > > > It seems the problem is that the json format provided by Calcite
> does
> > > not
> > > > > include properties specific to your operator?
> > > > > If so, I think you can override RelNode#explainTerms in your
> operator
> > > to
> > > > > attach the properties.
> > > > >
> > > > > Best,
> > > > > Liya Fan
> > > > >
> > > > > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <
> bhavya@knoldus.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Please let me know if we have an example for writing a custom
> > visitor
> > > > > that
> > > > > > I can use. I see a RelVisitor class that I can extend but how to
> > make
> > > > it
> > > > > to
> > > > > > visit the specific Physical Operator.
> > > > > >
> > > > > > Thanks and Regards
> > > > > > Bhavya
> > > > > >
> > > > > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <
> > bhavya@knoldus.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > I tried using the RelJsonWriter and tried to add my custom
> > > > > implementation
> > > > > > > in the my custom Rel Nodes but I am not getting all the data in
> > my
> > > > > child
> > > > > > > operators. We are trying to use Calcite for a columnar data and
> > we
> > > > need
> > > > > > the
> > > > > > > information of the projections in the TableScan as well because
> > we
> > > > need
> > > > > > to
> > > > > > > pass those projections to read specific columns but I am not
> able
> > > to
> > > > > find
> > > > > > > the projections in my Table scan. As per my understanding
> Calcite
> > > > > should
> > > > > > be
> > > > > > > passing this information as it is required for me reading the
> > > table.
> > > > > > Please
> > > > > > > let me know if I need to do something special to get that
> > > information
> > > > > in
> > > > > > > the child operator.
> > > > > > >
> > > > > > > Regards
> > > > > > > Bhavya
> > > > > > >
> > > > > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <
> mmior@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > >> That would likely be the best approach if you have some
> specific
> > > > JSON
> > > > > > >> format you're trying to generate. If you're happy with a JSON
> > > format
> > > > > > >> defined by Calcite, you can look at RelJsonWriter.
> > > > > > >>
> > > > > > >> --
> > > > > > >> Michael Mior
> > > > > > >> mmior@apache.org
> > > > > > >>
> > > > > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <
> > > mgelbana@apache.org
> > > > >
> > > > > a
> > > > > > >> écrit :
> > > > > > >> >
> > > > > > >> > I would use a visitor to traverse the optimized/physical
> plan.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> > > > bhavya@knoldus.com
> > > > > >
> > > > > > >> wrote:
> > > > > > >> >
> > > > > > >> > > Hi All,
> > > > > > >> > >
> > > > > > >> > > We need to generate a JSON object for the physical
> execution
> > > > tree
> > > > > > >> that has
> > > > > > >> > > been created. Is there an option in Calcite that we can
> use
> > to
> > > > do
> > > > > > >> this. I
> > > > > > >> > > am not sure what is the right approach to do it. Please
> let
> > me
> > > > > know
> > > > > > if
> > > > > > >> > > there are different ways to achieve this.
> > > > > > >> > >
> > > > > > >> > > Regards
> > > > > > >> > > Bhavya
> > > > > > >> > >
> > > > > > >> > > --
> > > > > > >> > > Your feedback matters - At Knoldus we aim to be very
> > > > professional
> > > > > in
> > > > > > >> our
> > > > > > >> > > quality of work, commitment to results, and proactive
> > > > > communication.
> > > > > > >> If
> > > > > > >> > > you
> > > > > > >> > > feel otherwise please share your feedback
> > > > > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work
> on
> > > it.
> > > > > > >> > >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Bhavya Aggarwal*
> > > > > > > CTO & Partner
> > > > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > > > +91-9910483067
> > > > > > > Canada - USA - India - Singapore
> > > > > > > <https://in.linkedin.com/company/knoldus> <
> > > > > https://twitter.com/Knolspeak
> > > > > > >
> > > > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > > > https://blog.knoldus.com/
> > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Bhavya Aggarwal*
> > > > > > CTO & Partner
> > > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > > +91-9910483067
> > > > > > Canada - USA - India - Singapore
> > > > > > <https://in.linkedin.com/company/knoldus> <
> > > > https://twitter.com/Knolspeak
> > > > > >
> > > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > > https://blog.knoldus.com/
> > > > >
> > > > > >
> > > > > > --
> > > > > > Your feedback matters - At Knoldus we aim to be very professional
> > in
> > > > our
> > > > > > quality of work, commitment to results, and proactive
> > communication.
> > > If
> > > > > > you
> > > > > > feel otherwise please share your feedback
> > > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > *Bhavya Aggarwal*
> > > > CTO & Partner
> > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > +91-9910483067
> > > > Canada - USA - India - Singapore
> > > > <https://in.linkedin.com/company/knoldus> <
> > https://twitter.com/Knolspeak
> > > >
> > > > <https://www.facebook.com/KnoldusSoftware/> <
> https://blog.knoldus.com/
> > >
> > > >
> > > > --
> > > > Your feedback matters - At Knoldus we aim to be very professional in
> > our
> > > > quality of work, commitment to results, and proactive communication.
> If
> > > > you
> > > > feel otherwise please share your feedback
> > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > >
> > >
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. <http://www.knoldus.com/>
> > +91-9910483067
> > Canada - USA - India - Singapore
> > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak
> >
> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
>
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Bhavya,
Thanks for your explanation.
I do not think a rule like ProjectFilterTableScan solves the problem.
To see this, please note that the parent operator of the LogicalProject may
need other columns from the table
(e.g. the parent operator may be a join, which uses some other columns as
the join key).
So the column pruning of table scan should be a global decision, depending
on the whole plan.
Calcite has RelMdColumnOrigins, which can help in this scenario.
In particular, given an output column in the output operator, it gives the
dependent columns in the underlying source tables.
So columns not referenced from the final output can be pruned from the
source table.
Best,
Liya Fan
On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Hi Liya,
>
> Please see below , I have a query as given below
>
> SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
>
> Now I know that I have to fetch two columns from my columnar database i.e.
> ID, Name, so need to pass these parameters in my custom Table Scan.
> But when I convert the LogicalTableScan to my custom Table scan I do not
> have that information, please see the tree below, in the Logical Table Scan
> the Input is null, I was hoping that I will have the columns in the
> LogicalTableScan, The question is what do I need to do in Calcite to pass
> that information to the LogicalTableScan or in my physical operator. The
> way I have found is that I create a Rule like ProjectFilterTableScan rule
> and then extract what information I need. Is there a better way to do this?
>
> -----------------Logical JSON Plan --------------------
> > {
> > "rels": [
> > {
> > "id": "0",
> > "relOp": "LogicalTableScan",
> > "table": [
> > "USERS"
> > ],
> > "inputs": []
> > },
> > {
> > "id": "1",
> > "relOp": "LogicalFilter",
> > "condition": {
> > "op": {
> > "name": "=",
> > "kind": "EQUALS",
> > "syntax": "BINARY"
> > },
> > "operands": [
> > {
> > "op": {
> > "name": "CAST",
> > "kind": "CAST",
> > "syntax": "SPECIAL"
> > },
> > "operands": [
> > {
> > "input": 1,
> > "name": "$1"
> > }
> > ],
> > "type": {
> > "type": "CHAR",
> > "nullable": true,
> > "precision": 5
> > }
> > },
> > {
> > "literal": "HELLO",
> > "type": {
> > "type": "CHAR",
> > "nullable": false,
> > "precision": 5
> > }
> > }
> > ]
> > }
> > },
> > {
> > "id": "2",
> > "relOp": "LogicalProject",
> > "fields": [
> > "ID"
> > ],
> > "exprs": [
> > {
> > "input": 0,
> > "name": "$0"
> > }
> > ]
> > }
> > ]
> > }
> >
> >> -----------------------------------------------------------
> >
> >
> Thanks and regards
> Bhavya
>
> On Fri, Dec 18, 2020 at 11:54 AM Fan Liya <li...@gmail.com> wrote:
>
> > Hi Bhavya,
> >
> > Sorry I do not understand your question. Why is it difficult to pass sth.
> > to child operators?
> >
> > Best,
> > Liya Fan
> >
> >
> > On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Hi Liya,
> > >
> > > Actually the question is how to pass the projection columns to Table
> Scan
> > > as right now in the LogicalTableScan there are no projection columns
> > being
> > > passed so when I am trying to create my custom JSON , I do not have
> those
> > > projected columns or columns that are being used in the query. I want
> to
> > > understand what is the calcite mechanism for passing it to child
> > operators
> > > without using Rules as it will be a lot of rule that we need to
> > implement.
> > >
> > > Thanks and Regards
> > > Bhavya
> > >
> > > On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com> wrote:
> > >
> > > > Hi Bhavya,
> > > >
> > > > It seems the problem is that the json format provided by Calcite does
> > not
> > > > include properties specific to your operator?
> > > > If so, I think you can override RelNode#explainTerms in your operator
> > to
> > > > attach the properties.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <bhavya@knoldus.com
> >
> > > > wrote:
> > > >
> > > > > Please let me know if we have an example for writing a custom
> visitor
> > > > that
> > > > > I can use. I see a RelVisitor class that I can extend but how to
> make
> > > it
> > > > to
> > > > > visit the specific Physical Operator.
> > > > >
> > > > > Thanks and Regards
> > > > > Bhavya
> > > > >
> > > > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <
> bhavya@knoldus.com>
> > > > > wrote:
> > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > I tried using the RelJsonWriter and tried to add my custom
> > > > implementation
> > > > > > in the my custom Rel Nodes but I am not getting all the data in
> my
> > > > child
> > > > > > operators. We are trying to use Calcite for a columnar data and
> we
> > > need
> > > > > the
> > > > > > information of the projections in the TableScan as well because
> we
> > > need
> > > > > to
> > > > > > pass those projections to read specific columns but I am not able
> > to
> > > > find
> > > > > > the projections in my Table scan. As per my understanding Calcite
> > > > should
> > > > > be
> > > > > > passing this information as it is required for me reading the
> > table.
> > > > > Please
> > > > > > let me know if I need to do something special to get that
> > information
> > > > in
> > > > > > the child operator.
> > > > > >
> > > > > > Regards
> > > > > > Bhavya
> > > > > >
> > > > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org>
> > > > wrote:
> > > > > >
> > > > > >> That would likely be the best approach if you have some specific
> > > JSON
> > > > > >> format you're trying to generate. If you're happy with a JSON
> > format
> > > > > >> defined by Calcite, you can look at RelJsonWriter.
> > > > > >>
> > > > > >> --
> > > > > >> Michael Mior
> > > > > >> mmior@apache.org
> > > > > >>
> > > > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <
> > mgelbana@apache.org
> > > >
> > > > a
> > > > > >> écrit :
> > > > > >> >
> > > > > >> > I would use a visitor to traverse the optimized/physical plan.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> > > bhavya@knoldus.com
> > > > >
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > > Hi All,
> > > > > >> > >
> > > > > >> > > We need to generate a JSON object for the physical execution
> > > tree
> > > > > >> that has
> > > > > >> > > been created. Is there an option in Calcite that we can use
> to
> > > do
> > > > > >> this. I
> > > > > >> > > am not sure what is the right approach to do it. Please let
> me
> > > > know
> > > > > if
> > > > > >> > > there are different ways to achieve this.
> > > > > >> > >
> > > > > >> > > Regards
> > > > > >> > > Bhavya
> > > > > >> > >
> > > > > >> > > --
> > > > > >> > > Your feedback matters - At Knoldus we aim to be very
> > > professional
> > > > in
> > > > > >> our
> > > > > >> > > quality of work, commitment to results, and proactive
> > > > communication.
> > > > > >> If
> > > > > >> > > you
> > > > > >> > > feel otherwise please share your feedback
> > > > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on
> > it.
> > > > > >> > >
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Bhavya Aggarwal*
> > > > > > CTO & Partner
> > > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > > +91-9910483067
> > > > > > Canada - USA - India - Singapore
> > > > > > <https://in.linkedin.com/company/knoldus> <
> > > > https://twitter.com/Knolspeak
> > > > > >
> > > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > > https://blog.knoldus.com/
> > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Bhavya Aggarwal*
> > > > > CTO & Partner
> > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > +91-9910483067
> > > > > Canada - USA - India - Singapore
> > > > > <https://in.linkedin.com/company/knoldus> <
> > > https://twitter.com/Knolspeak
> > > > >
> > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > https://blog.knoldus.com/
> > > >
> > > > >
> > > > > --
> > > > > Your feedback matters - At Knoldus we aim to be very professional
> in
> > > our
> > > > > quality of work, commitment to results, and proactive
> communication.
> > If
> > > > > you
> > > > > feel otherwise please share your feedback
> > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Hi Liya,
Please see below , I have a query as given below
SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
Now I know that I have to fetch two columns from my columnar database i.e.
ID, Name, so need to pass these parameters in my custom Table Scan.
But when I convert the LogicalTableScan to my custom Table scan I do not
have that information, please see the tree below, in the Logical Table Scan
the Input is null, I was hoping that I will have the columns in the
LogicalTableScan, The question is what do I need to do in Calcite to pass
that information to the LogicalTableScan or in my physical operator. The
way I have found is that I create a Rule like ProjectFilterTableScan rule
and then extract what information I need. Is there a better way to do this?
-----------------Logical JSON Plan --------------------
> {
> "rels": [
> {
> "id": "0",
> "relOp": "LogicalTableScan",
> "table": [
> "USERS"
> ],
> "inputs": []
> },
> {
> "id": "1",
> "relOp": "LogicalFilter",
> "condition": {
> "op": {
> "name": "=",
> "kind": "EQUALS",
> "syntax": "BINARY"
> },
> "operands": [
> {
> "op": {
> "name": "CAST",
> "kind": "CAST",
> "syntax": "SPECIAL"
> },
> "operands": [
> {
> "input": 1,
> "name": "$1"
> }
> ],
> "type": {
> "type": "CHAR",
> "nullable": true,
> "precision": 5
> }
> },
> {
> "literal": "HELLO",
> "type": {
> "type": "CHAR",
> "nullable": false,
> "precision": 5
> }
> }
> ]
> }
> },
> {
> "id": "2",
> "relOp": "LogicalProject",
> "fields": [
> "ID"
> ],
> "exprs": [
> {
> "input": 0,
> "name": "$0"
> }
> ]
> }
> ]
> }
>
>> -----------------------------------------------------------
>
>
Thanks and regards
Bhavya
On Fri, Dec 18, 2020 at 11:54 AM Fan Liya <li...@gmail.com> wrote:
> Hi Bhavya,
>
> Sorry I do not understand your question. Why is it difficult to pass sth.
> to child operators?
>
> Best,
> Liya Fan
>
>
> On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Hi Liya,
> >
> > Actually the question is how to pass the projection columns to Table Scan
> > as right now in the LogicalTableScan there are no projection columns
> being
> > passed so when I am trying to create my custom JSON , I do not have those
> > projected columns or columns that are being used in the query. I want to
> > understand what is the calcite mechanism for passing it to child
> operators
> > without using Rules as it will be a lot of rule that we need to
> implement.
> >
> > Thanks and Regards
> > Bhavya
> >
> > On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com> wrote:
> >
> > > Hi Bhavya,
> > >
> > > It seems the problem is that the json format provided by Calcite does
> not
> > > include properties specific to your operator?
> > > If so, I think you can override RelNode#explainTerms in your operator
> to
> > > attach the properties.
> > >
> > > Best,
> > > Liya Fan
> > >
> > > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > wrote:
> > >
> > > > Please let me know if we have an example for writing a custom visitor
> > > that
> > > > I can use. I see a RelVisitor class that I can extend but how to make
> > it
> > > to
> > > > visit the specific Physical Operator.
> > > >
> > > > Thanks and Regards
> > > > Bhavya
> > > >
> > > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > > wrote:
> > > >
> > > > > Thanks,
> > > > >
> > > > > I tried using the RelJsonWriter and tried to add my custom
> > > implementation
> > > > > in the my custom Rel Nodes but I am not getting all the data in my
> > > child
> > > > > operators. We are trying to use Calcite for a columnar data and we
> > need
> > > > the
> > > > > information of the projections in the TableScan as well because we
> > need
> > > > to
> > > > > pass those projections to read specific columns but I am not able
> to
> > > find
> > > > > the projections in my Table scan. As per my understanding Calcite
> > > should
> > > > be
> > > > > passing this information as it is required for me reading the
> table.
> > > > Please
> > > > > let me know if I need to do something special to get that
> information
> > > in
> > > > > the child operator.
> > > > >
> > > > > Regards
> > > > > Bhavya
> > > > >
> > > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org>
> > > wrote:
> > > > >
> > > > >> That would likely be the best approach if you have some specific
> > JSON
> > > > >> format you're trying to generate. If you're happy with a JSON
> format
> > > > >> defined by Calcite, you can look at RelJsonWriter.
> > > > >>
> > > > >> --
> > > > >> Michael Mior
> > > > >> mmior@apache.org
> > > > >>
> > > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <
> mgelbana@apache.org
> > >
> > > a
> > > > >> écrit :
> > > > >> >
> > > > >> > I would use a visitor to traverse the optimized/physical plan.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> > bhavya@knoldus.com
> > > >
> > > > >> wrote:
> > > > >> >
> > > > >> > > Hi All,
> > > > >> > >
> > > > >> > > We need to generate a JSON object for the physical execution
> > tree
> > > > >> that has
> > > > >> > > been created. Is there an option in Calcite that we can use to
> > do
> > > > >> this. I
> > > > >> > > am not sure what is the right approach to do it. Please let me
> > > know
> > > > if
> > > > >> > > there are different ways to achieve this.
> > > > >> > >
> > > > >> > > Regards
> > > > >> > > Bhavya
> > > > >> > >
> > > > >> > > --
> > > > >> > > Your feedback matters - At Knoldus we aim to be very
> > professional
> > > in
> > > > >> our
> > > > >> > > quality of work, commitment to results, and proactive
> > > communication.
> > > > >> If
> > > > >> > > you
> > > > >> > > feel otherwise please share your feedback
> > > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on
> it.
> > > > >> > >
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > *Bhavya Aggarwal*
> > > > > CTO & Partner
> > > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > > +91-9910483067
> > > > > Canada - USA - India - Singapore
> > > > > <https://in.linkedin.com/company/knoldus> <
> > > https://twitter.com/Knolspeak
> > > > >
> > > > > <https://www.facebook.com/KnoldusSoftware/> <
> > https://blog.knoldus.com/
> > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > *Bhavya Aggarwal*
> > > > CTO & Partner
> > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > +91-9910483067
> > > > Canada - USA - India - Singapore
> > > > <https://in.linkedin.com/company/knoldus> <
> > https://twitter.com/Knolspeak
> > > >
> > > > <https://www.facebook.com/KnoldusSoftware/> <
> https://blog.knoldus.com/
> > >
> > > >
> > > > --
> > > > Your feedback matters - At Knoldus we aim to be very professional in
> > our
> > > > quality of work, commitment to results, and proactive communication.
> If
> > > > you
> > > > feel otherwise please share your feedback
> > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > >
> > >
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. <http://www.knoldus.com/>
> > +91-9910483067
> > Canada - USA - India - Singapore
> > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak
> >
> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
>
--
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. <http://www.knoldus.com/>
+91-9910483067
Canada - USA - India - Singapore
<https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
<https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Bhavya,
Sorry I do not understand your question. Why is it difficult to pass sth.
to child operators?
Best,
Liya Fan
On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Hi Liya,
>
> Actually the question is how to pass the projection columns to Table Scan
> as right now in the LogicalTableScan there are no projection columns being
> passed so when I am trying to create my custom JSON , I do not have those
> projected columns or columns that are being used in the query. I want to
> understand what is the calcite mechanism for passing it to child operators
> without using Rules as it will be a lot of rule that we need to implement.
>
> Thanks and Regards
> Bhavya
>
> On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com> wrote:
>
> > Hi Bhavya,
> >
> > It seems the problem is that the json format provided by Calcite does not
> > include properties specific to your operator?
> > If so, I think you can override RelNode#explainTerms in your operator to
> > attach the properties.
> >
> > Best,
> > Liya Fan
> >
> > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Please let me know if we have an example for writing a custom visitor
> > that
> > > I can use. I see a RelVisitor class that I can extend but how to make
> it
> > to
> > > visit the specific Physical Operator.
> > >
> > > Thanks and Regards
> > > Bhavya
> > >
> > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> > > wrote:
> > >
> > > > Thanks,
> > > >
> > > > I tried using the RelJsonWriter and tried to add my custom
> > implementation
> > > > in the my custom Rel Nodes but I am not getting all the data in my
> > child
> > > > operators. We are trying to use Calcite for a columnar data and we
> need
> > > the
> > > > information of the projections in the TableScan as well because we
> need
> > > to
> > > > pass those projections to read specific columns but I am not able to
> > find
> > > > the projections in my Table scan. As per my understanding Calcite
> > should
> > > be
> > > > passing this information as it is required for me reading the table.
> > > Please
> > > > let me know if I need to do something special to get that information
> > in
> > > > the child operator.
> > > >
> > > > Regards
> > > > Bhavya
> > > >
> > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org>
> > wrote:
> > > >
> > > >> That would likely be the best approach if you have some specific
> JSON
> > > >> format you're trying to generate. If you're happy with a JSON format
> > > >> defined by Calcite, you can look at RelJsonWriter.
> > > >>
> > > >> --
> > > >> Michael Mior
> > > >> mmior@apache.org
> > > >>
> > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mgelbana@apache.org
> >
> > a
> > > >> écrit :
> > > >> >
> > > >> > I would use a visitor to traverse the optimized/physical plan.
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> bhavya@knoldus.com
> > >
> > > >> wrote:
> > > >> >
> > > >> > > Hi All,
> > > >> > >
> > > >> > > We need to generate a JSON object for the physical execution
> tree
> > > >> that has
> > > >> > > been created. Is there an option in Calcite that we can use to
> do
> > > >> this. I
> > > >> > > am not sure what is the right approach to do it. Please let me
> > know
> > > if
> > > >> > > there are different ways to achieve this.
> > > >> > >
> > > >> > > Regards
> > > >> > > Bhavya
> > > >> > >
> > > >> > > --
> > > >> > > Your feedback matters - At Knoldus we aim to be very
> professional
> > in
> > > >> our
> > > >> > > quality of work, commitment to results, and proactive
> > communication.
> > > >> If
> > > >> > > you
> > > >> > > feel otherwise please share your feedback
> > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > >> > >
> > > >>
> > > >
> > > >
> > > > --
> > > > *Bhavya Aggarwal*
> > > > CTO & Partner
> > > > Knoldus Inc. <http://www.knoldus.com/>
> > > > +91-9910483067
> > > > Canada - USA - India - Singapore
> > > > <https://in.linkedin.com/company/knoldus> <
> > https://twitter.com/Knolspeak
> > > >
> > > > <https://www.facebook.com/KnoldusSoftware/> <
> https://blog.knoldus.com/
> > >
> > > >
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Hi Liya,
Actually the question is how to pass the projection columns to Table Scan
as right now in the LogicalTableScan there are no projection columns being
passed so when I am trying to create my custom JSON , I do not have those
projected columns or columns that are being used in the query. I want to
understand what is the calcite mechanism for passing it to child operators
without using Rules as it will be a lot of rule that we need to implement.
Thanks and Regards
Bhavya
On Fri, Dec 18, 2020 at 9:47 AM Fan Liya <li...@gmail.com> wrote:
> Hi Bhavya,
>
> It seems the problem is that the json format provided by Calcite does not
> include properties specific to your operator?
> If so, I think you can override RelNode#explainTerms in your operator to
> attach the properties.
>
> Best,
> Liya Fan
>
> On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Please let me know if we have an example for writing a custom visitor
> that
> > I can use. I see a RelVisitor class that I can extend but how to make it
> to
> > visit the specific Physical Operator.
> >
> > Thanks and Regards
> > Bhavya
> >
> > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> > wrote:
> >
> > > Thanks,
> > >
> > > I tried using the RelJsonWriter and tried to add my custom
> implementation
> > > in the my custom Rel Nodes but I am not getting all the data in my
> child
> > > operators. We are trying to use Calcite for a columnar data and we need
> > the
> > > information of the projections in the TableScan as well because we need
> > to
> > > pass those projections to read specific columns but I am not able to
> find
> > > the projections in my Table scan. As per my understanding Calcite
> should
> > be
> > > passing this information as it is required for me reading the table.
> > Please
> > > let me know if I need to do something special to get that information
> in
> > > the child operator.
> > >
> > > Regards
> > > Bhavya
> > >
> > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org>
> wrote:
> > >
> > >> That would likely be the best approach if you have some specific JSON
> > >> format you're trying to generate. If you're happy with a JSON format
> > >> defined by Calcite, you can look at RelJsonWriter.
> > >>
> > >> --
> > >> Michael Mior
> > >> mmior@apache.org
> > >>
> > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mg...@apache.org>
> a
> > >> écrit :
> > >> >
> > >> > I would use a visitor to traverse the optimized/physical plan.
> > >> >
> > >> >
> > >> >
> > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bhavya@knoldus.com
> >
> > >> wrote:
> > >> >
> > >> > > Hi All,
> > >> > >
> > >> > > We need to generate a JSON object for the physical execution tree
> > >> that has
> > >> > > been created. Is there an option in Calcite that we can use to do
> > >> this. I
> > >> > > am not sure what is the right approach to do it. Please let me
> know
> > if
> > >> > > there are different ways to achieve this.
> > >> > >
> > >> > > Regards
> > >> > > Bhavya
> > >> > >
> > >> > > --
> > >> > > Your feedback matters - At Knoldus we aim to be very professional
> in
> > >> our
> > >> > > quality of work, commitment to results, and proactive
> communication.
> > >> If
> > >> > > you
> > >> > > feel otherwise please share your feedback
> > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >> > >
> > >>
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. <http://www.knoldus.com/>
> > +91-9910483067
> > Canada - USA - India - Singapore
> > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak
> >
> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
>
--
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. <http://www.knoldus.com/>
+91-9910483067
Canada - USA - India - Singapore
<https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
<https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Fan Liya <li...@gmail.com>.
Hi Bhavya,
It seems the problem is that the json format provided by Calcite does not
include properties specific to your operator?
If so, I think you can override RelNode#explainTerms in your operator to
attach the properties.
Best,
Liya Fan
On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Please let me know if we have an example for writing a custom visitor that
> I can use. I see a RelVisitor class that I can extend but how to make it to
> visit the specific Physical Operator.
>
> Thanks and Regards
> Bhavya
>
> On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
>
> > Thanks,
> >
> > I tried using the RelJsonWriter and tried to add my custom implementation
> > in the my custom Rel Nodes but I am not getting all the data in my child
> > operators. We are trying to use Calcite for a columnar data and we need
> the
> > information of the projections in the TableScan as well because we need
> to
> > pass those projections to read specific columns but I am not able to find
> > the projections in my Table scan. As per my understanding Calcite should
> be
> > passing this information as it is required for me reading the table.
> Please
> > let me know if I need to do something special to get that information in
> > the child operator.
> >
> > Regards
> > Bhavya
> >
> > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org> wrote:
> >
> >> That would likely be the best approach if you have some specific JSON
> >> format you're trying to generate. If you're happy with a JSON format
> >> defined by Calcite, you can look at RelJsonWriter.
> >>
> >> --
> >> Michael Mior
> >> mmior@apache.org
> >>
> >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mg...@apache.org> a
> >> écrit :
> >> >
> >> > I would use a visitor to traverse the optimized/physical plan.
> >> >
> >> >
> >> >
> >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bh...@knoldus.com>
> >> wrote:
> >> >
> >> > > Hi All,
> >> > >
> >> > > We need to generate a JSON object for the physical execution tree
> >> that has
> >> > > been created. Is there an option in Calcite that we can use to do
> >> this. I
> >> > > am not sure what is the right approach to do it. Please let me know
> if
> >> > > there are different ways to achieve this.
> >> > >
> >> > > Regards
> >> > > Bhavya
> >> > >
> >> > > --
> >> > > Your feedback matters - At Knoldus we aim to be very professional in
> >> our
> >> > > quality of work, commitment to results, and proactive communication.
> >> If
> >> > > you
> >> > > feel otherwise please share your feedback
> >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >> > >
> >>
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. <http://www.knoldus.com/>
> > +91-9910483067
> > Canada - USA - India - Singapore
> > <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak
> >
> > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Please let me know if we have an example for writing a custom visitor that
I can use. I see a RelVisitor class that I can extend but how to make it to
visit the specific Physical Operator.
Thanks and Regards
Bhavya
On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Thanks,
>
> I tried using the RelJsonWriter and tried to add my custom implementation
> in the my custom Rel Nodes but I am not getting all the data in my child
> operators. We are trying to use Calcite for a columnar data and we need the
> information of the projections in the TableScan as well because we need to
> pass those projections to read specific columns but I am not able to find
> the projections in my Table scan. As per my understanding Calcite should be
> passing this information as it is required for me reading the table. Please
> let me know if I need to do something special to get that information in
> the child operator.
>
> Regards
> Bhavya
>
> On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org> wrote:
>
>> That would likely be the best approach if you have some specific JSON
>> format you're trying to generate. If you're happy with a JSON format
>> defined by Calcite, you can look at RelJsonWriter.
>>
>> --
>> Michael Mior
>> mmior@apache.org
>>
>> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mg...@apache.org> a
>> écrit :
>> >
>> > I would use a visitor to traverse the optimized/physical plan.
>> >
>> >
>> >
>> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bh...@knoldus.com>
>> wrote:
>> >
>> > > Hi All,
>> > >
>> > > We need to generate a JSON object for the physical execution tree
>> that has
>> > > been created. Is there an option in Calcite that we can use to do
>> this. I
>> > > am not sure what is the right approach to do it. Please let me know if
>> > > there are different ways to achieve this.
>> > >
>> > > Regards
>> > > Bhavya
>> > >
>> > > --
>> > > Your feedback matters - At Knoldus we aim to be very professional in
>> our
>> > > quality of work, commitment to results, and proactive communication.
>> If
>> > > you
>> > > feel otherwise please share your feedback
>> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>> > >
>>
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
--
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. <http://www.knoldus.com/>
+91-9910483067
Canada - USA - India - Singapore
<https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
<https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Bhavya Aggarwal <bh...@knoldus.com>.
Thanks,
I tried using the RelJsonWriter and tried to add my custom implementation
in the my custom Rel Nodes but I am not getting all the data in my child
operators. We are trying to use Calcite for a columnar data and we need the
information of the projections in the TableScan as well because we need to
pass those projections to read specific columns but I am not able to find
the projections in my Table scan. As per my understanding Calcite should be
passing this information as it is required for me reading the table. Please
let me know if I need to do something special to get that information in
the child operator.
Regards
Bhavya
On Sun, Dec 13, 2020 at 11:32 PM Michael Mior <mm...@apache.org> wrote:
> That would likely be the best approach if you have some specific JSON
> format you're trying to generate. If you're happy with a JSON format
> defined by Calcite, you can look at RelJsonWriter.
>
> --
> Michael Mior
> mmior@apache.org
>
> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mg...@apache.org> a
> écrit :
> >
> > I would use a visitor to traverse the optimized/physical plan.
> >
> >
> >
> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bh...@knoldus.com>
> wrote:
> >
> > > Hi All,
> > >
> > > We need to generate a JSON object for the physical execution tree that
> has
> > > been created. Is there an option in Calcite that we can use to do
> this. I
> > > am not sure what is the right approach to do it. Please let me know if
> > > there are different ways to achieve this.
> > >
> > > Regards
> > > Bhavya
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
>
--
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. <http://www.knoldus.com/>
+91-9910483067
Canada - USA - India - Singapore
<https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
<https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
--
Your feedback matters - At Knoldus we aim to be very professional in our
quality of work, commitment to results, and proactive communication. If you
feel otherwise please share your feedback
<https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
Re: Need a Jason output if physical tree
Posted by Michael Mior <mm...@apache.org>.
That would likely be the best approach if you have some specific JSON
format you're trying to generate. If you're happy with a JSON format
defined by Calcite, you can look at RelJsonWriter.
--
Michael Mior
mmior@apache.org
Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana <mg...@apache.org> a écrit :
>
> I would use a visitor to traverse the optimized/physical plan.
>
>
>
> On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bh...@knoldus.com> wrote:
>
> > Hi All,
> >
> > We need to generate a JSON object for the physical execution tree that has
> > been created. Is there an option in Calcite that we can use to do this. I
> > am not sure what is the right approach to do it. Please let me know if
> > there are different ways to achieve this.
> >
> > Regards
> > Bhavya
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> >
Re: Need a Jason output if physical tree
Posted by Muhammad Gelbana <mg...@apache.org>.
I would use a visitor to traverse the optimized/physical plan.
On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <bh...@knoldus.com> wrote:
> Hi All,
>
> We need to generate a JSON object for the physical execution tree that has
> been created. Is there an option in Calcite that we can use to do this. I
> am not sure what is the right approach to do it. Please let me know if
> there are different ways to achieve this.
>
> Regards
> Bhavya
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>