Materialized Views and Lattices do not work in a multi-threaded application

[Rajat Venkatesh <rv...@qubole.com>]

Quark JDBC uses Materialization service to setup Materialized views and
Lattices in the "connect" function in calcite. Calcite optimizes queries
correctly in an application like sqlLine which is single threaded.

However, these optimizations are not triggered in multi-threaded
applications like Apache Zeppelin. Quark and Calcite is not used
concurrently but it is owned by different threads at different times. For
e.g. in
https://github.com/apache/incubator-zeppelin/blob/master/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java,
JDBC connect is called by one thread and statements are executed by another
thread. The same is true in an internal service we are implementing at
Qubole.

Right now, we have worked around this problem by managing the life cycle of
the MaterializationService object. Check this commit:
https://github.com/qubole/quark/commit/28141fff618cdc20113f17ac8932b0e137868b85
This
approach doesnt feel right.

I have a couple of questions:
i) Are we using Calcite incorrectly ? To me MaterializationService object
belongs to the Planner object. There must be a good reason why it isnt so.
ii) Are others facing similar issues ? For e.g. Apache Phoenix also uses
materialized views. How did you get it to work in multi-threaded apps like
Zeppelin.