Intended behaviour in the HepPlanner union rewriting

[Mark Pasterkamp <ma...@gmail.com>]

Dear all,

For some time now I have been getting issues with getiing materialized view
union rewriting to work using the HepPlanner.
A few weeks ago I have been able to find the culprit and I was wondering
whether this is supposed to be intended behaviour.

in [1] the code suggests that you go through all parents in the query DAG,
so any circular reference  should be seen at this point. But what it does
is it only looks at the direct parents of the node if they are exactly the
same.

In my case, I had an issue where the parent of that parent (so say a
sequence of node A - node B - node A) was exactly the same node, which
caused issues.

Would you instead return all parents you can then check if the current node
is already contained in the parent list and if that is the case, you can
just return the current node resolving the circular reference.

I don't know if this is intended behaviour of the HepPlanner or not, but I
thought it might be useful to report my findings in case someone else in
the future will stumble upon this issue.


Mark

[1]
https://github.com/apache/calcite/blob/8c0d792dd735eded2bb11be0c7c195b8f302ae90/core/src/main/java/org/apache/calcite/plan/hep/HepPlanner.java#L723

Re: Intended behaviour in the HepPlanner union rewriting

[Julian Hyde <jh...@apache.org>]

I don’t think we ever considered that case.

Can you please log it, so that it is on record.

I haven’t analyzed the issue, but my instinct is that it is a reasonable behavior to want and expect, and that it is not that difficult to fix, and if so, someone should fix it.

Julian


> On Jul 17, 2019, at 2:29 AM, Mark Pasterkamp <ma...@gmail.com> wrote:
> 
> Dear all,
> 
> For some time now I have been getting issues with getiing materialized view
> union rewriting to work using the HepPlanner.
> A few weeks ago I have been able to find the culprit and I was wondering
> whether this is supposed to be intended behaviour.
> 
> in [1] the code suggests that you go through all parents in the query DAG,
> so any circular reference  should be seen at this point. But what it does
> is it only looks at the direct parents of the node if they are exactly the
> same.
> 
> In my case, I had an issue where the parent of that parent (so say a
> sequence of node A - node B - node A) was exactly the same node, which
> caused issues.
> 
> Would you instead return all parents you can then check if the current node
> is already contained in the parent list and if that is the case, you can
> just return the current node resolving the circular reference.
> 
> I don't know if this is intended behaviour of the HepPlanner or not, but I
> thought it might be useful to report my findings in case someone else in
> the future will stumble upon this issue.
> 
> 
> Mark
> 
> [1]
> https://github.com/apache/calcite/blob/8c0d792dd735eded2bb11be0c7c195b8f302ae90/core/src/main/java/org/apache/calcite/plan/hep/HepPlanner.java#L723