Apache Pinot Daily Email Digest (2020-10-13)

[Pinot Slack Email Digest <sn...@apache.org>]

### _#general_

  
 **@matthew.park:** @matthew.park has joined the channel  

###  _#random_

  
 **@matthew.park:** @matthew.park has joined the channel  

###  _#troubleshooting_

  
 **@pradeepgv42:** Hi, for the hybrid tables is there a way to adjust the
boundary used by the broker for querying offline vs realtime?  
**@mayanks:** I think there's a proposal for that. cc @moradi.sajjad  
**@pradeepgv42:** nice thanks @mayanks let me know if there is tracker I can
follow for this  
**@mayanks:**  
**@pradeepgv42:** thanks  
**@mayanks:** And the issue:  

### _#lp-pinot-poc_

  
 **@andrew:** this is the result with those settings. still degrades after a
couple of hours  
 **@andrew:**  
 **@andrew:**  
 **@andrew:**  
 **@fx19880617:** hmm  
 **@fx19880617:** have you seen data movement happens to offline servers?  
 **@fx19880617:** can you break that number of segments in Table to be server
level metrics  

###  _#pql-sql-regression_

  
 **@steotia:** I would like to get an opinion on a slightly related topic.
Query rewrite code has grown in non-trivial manner and I believe it will
continue to grow. Can we add a fast path query compilation flag to bypass
query rewrite? On the SQL path l, there are few different things happening in
compilation 1\. Calcite parsing to SQLNode 2\. SQLNode to PinotQuery 3\.
PinotQuery rewrite.. 4\. PinotQuery to BrokerRequest We should get rid of 4
very soon. In addition to that we should make 3 optional. Enabled by default.
We can't do much about 1 except tuning config and making changes to calcite. 2
may have some room.. Request compilation metric used to stay close to 1ms or
900us for PQL. For SQL, I have seen it go up to 5ms.  
 **@mayanks:** Do you have the profile for 5ms?  
 **@mayanks:** What part of it is coming from rewrite  
 **@steotia:** I can get the profile. It was an immediate thought after
looking at the time and the code.  
 **@mayanks:** Ok, let’s have that. In general, I’d like to avoid configs that
require tuning, or setting to get around some problem, we should find a way to
automatically do that inside the code  
 **@g.kishore:** 5ms is definitely a lot  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org