You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Feinauer <j....@pragmaticminds.de> on 2019/07/30 14:42:31 UTC

Request for Review: One step towards MATCH_RECOGNIZE

Hi all,

I finally finished to bring the joint work on MATCH_RECOGNIZE to a state where at least two non-trivial Tests work, see [3].
The Work is based on a lot of preliminary work of Hongze and Julian (Hyde) which was done over a period of over a year, therefore the code is rather large.
I also decided to not squash this PR (yet) as most of the code is not from myself but from Hongze and Julian which would be lost, in case of a squash.
As I had some issues during the implementation and found, I think, some bugs in (yet) unused parts of the code I would be very grateful for support with reviewing this PR and bringing the code base to a state where it is merge-able into master.

Most of the discussions can be found in [1] and [2].

The tests that work can be found in JdbcTest:

  *   testSimpleMatch
  *   testMatch

The query that works now is:

```
select *
from "hr"."emps" match_recognize (
  order by "empid" desc
  measures "commission" as c,
    "empid" as empid
  pattern (s up)
  define up as up."commission" < prev(up."commission"))
```
which covers all basic ingredients of the MATCH_RECOGNIZE clause.

The PR can be found in [4].

This PR does NOT yet completely resolve CALCITE-1935 as the given match.iq file does not yet work but I think it is a good idea to resync it with mainline and fix some flaws in my code before moving on.

As you may know I made only very few contributions to the Calcite codebase yet, so please forgive me if some of my approaches in the Code are rather unusual or bad design.

If there are any questions regarding my implementation please feel free to discuss.
I think this PR sets the first MWE for Match recognize and can be the basis for all the other (missing) features.

As this branch is pretty old was worked on by several people I do not know if all changes are reasonable, so it would be great if original authos (Julian, Hongze) could look into these diffs.
Short list of things reviewers should look into:

  *   RelBuilder – Don’t know about those changes?
  *   CircularArrayList – Is unused I think and at least the Tests had huge performance issues, could be removed, I think
  *   blank.iq – CoreQuidemTest fails and I have no idea why as I see no changes around. Probably that has to do with ExtensionSqlParser (?)
  *   I’m unsure about my changes in RexImpTable and would like to get comments about that
  *   Match.java:197 – I had to introduce this (ugly?) hack to make the tests in JdbcTest work. Perhaps someone could me help with that and explain why the former line fails?
  *   RexAction / RexPattern – I have no idea about those files and if they can be removed?

If there are any further questions please feel free to ask.

Best
Julian

[1] https://lists.apache.org/thread.html/f36852ea8fd49419da1492ee8c15a0cd0dd205f37000de0fe0032d32@%3Cdev.calcite.apache.org%3E
[2] https://lists.apache.org/thread.html/8aa11cb17a0a7c6ebc420f1a025e7521aa9d9cea56de1513b27b6c5b@%3Cdev.calcite.apache.org%3E
[3] https://issues.apache.org/jira/browse/CALCITE-1935
[4] https://github.com/apache/calcite/pull/1343