[GitHub] [incubator-pinot] siddharthteotia opened a new pull request #4901: Use FilterQuery instead of FilterQueryTree in optimizer.

[GitBox <gi...@apache.org>]

siddharthteotia opened a new pull request #4901: Use FilterQuery instead of FilterQueryTree in optimizer.
URL: https://github.com/apache/incubator-pinot/pull/4901
 
 
   When we build the BrokerRequest from AST, for the WHERE clause we do the following:
   
   (1) Build FilterQueryTree (this is a non-serializable object)
   (2) Build FilterQuery from FilterQuery -- this is thrift compliant object and is sent to servers for execution as part of BrokerRequest.
   
   On the server, we convert the received FilterQuery to FilterQueryTree and use that as a basis to construct physical filter operators.
   
   So there is essentially a conversion of FilterQueryTree  -> FilterQuery on the broker and vice-versa on the server during execution.
   
   However, the broker as part of optimization converts the FilterQuery back to FilterQueryTree, runs the optimizer on latter, converts it back to FilterQuery and updates the FilterQuery in broker request. This is a needless conversion of FilterQueryTree <=> FilterQuery since the optimization could have directly been done on the FiterQuery.
   
   This PR adds support for avoiding the extra conversion of FilterQueryTree <=> FilterQuery at the broker for optimization.
   
   There are currently 3 filter query tree optimizers:
   
   MultipleOrInequalitiesToInClause
   RangeMerge
   Flatten
   
   All these are invoked by BrokerRequestOptimize. This PR adds a new version of the first one to directly work on FilterQuery. Also addresses some other cases where the rewrite was not being done in the nested predicates. Secondly, it also addresses the rewrite of single value IN predicate to equality -- https://github.com/apache/incubator-pinot/pull/4847
   
   The new version is currently being used only in the test code. For the ease of review, I split the work so that when the follow-up PRs come and we have the new versions of other 2 filter query tree optimizers as well then we can simply integrate them into BrokerRequestOptimizer (simple change) and delete the 3 old optimizers. 
   
   
   At the broker, as part of optimization phase, we conver
   
   Doing step by step taking one filter query optimizer at a time.
   This PR takes the MultipleOrEqualitiesToInClauseFilterQueryTreeOptimzer
   and writes another version of it as MultiplOrEqualitiesToFilterQueryOptimizer.
   
   The new version operates on FilterQuery directly.
   
   Once the remaining two (Flatten and RangeMerge) are also rewritten,
   we will integrate them into BrokerRequestOptimizer and removed the 3 old
   optimizers.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org