Apache Pinot Daily Email Digest (2022-05-04)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@weixiang.sun:** What is the difference between timeColumnName and
sortedColumn inside the tableConfig from query performance perspective? If my
query is mainly based on timeColumnName, should I use use the same column as
sortedColumn?  
**@mayanks:** Typically you want to use a column that has `where col = xxx` in
most queries and that prunes out a lot of data from being selected. I take it
that you will have time range in your query like `where time between t1 and
t2`? If so, range index might be better fit for that. Also, there is time
related metadata that is used for pruning data during query execution.  
 **@buntyk91:** Hi All, Is it possible to launch pinot cluster on kubernetes
and point it to an already deployed zookeeper consisting of 1 month old
metadata.This is more of migration of all the components except zookeeper to
another kubernetes cluster within the same VPN.  
**@mayanks:** If metadata exists, and also the data exists in the same deep
store, and the instance names are the same, then yes  
**@xiangfu0:** For k8s, you need to make sure the deployment size and all the
pod names are identical from old Pinot cluster. Also curious how do you make
external zk available for k8s?  
 **@saumya2700:** Hi All, if we increase kafka partition later as per
requirement how pinot will behave and do we we need to change some config to
tackle this situation in pinot to avoid any issues or it is fine pinot will
create new segments as soon as the new partitions will be added to kafka
topics?  
**@kharekartik:** Hi, that's not an issue at all. Pinot automatically detects
new partitions in the topic. It checks for new partitions whenever
`RealtimeSegmentValidationManager` perioidic job runs and start consumers for
new partitions. You can configure the interval for this job using
`controller.realtime.segment.validation.frequencyPeriod` in controller
configuration;  
**@kharekartik:** You can also trigger this task manually using `curl -X GET
"" -H "accept: application/json"` Where you need to replace host:port with
your controller host:port and `your-table-name` with appropriate value  
**@saumya2700:** Thanks much Kartik this is really helpful  
 **@karinwolok1:** Just a reminder! :mega: *StarTree's FIRST in-person
conference is scheduled and we're looking for speakers!!!* :mega: *Real Time
Analytics Summit* (August 16/17 in San Francisco) You can submit a session or
register here:  Sponsorship opps also available. If interested, please shoot
me a message! :slightly_smiling_face:  
 **@deepakkumar05.it:** @deepakkumar05.it has joined the channel  
 **@xiangfu0:** Dear Community, TL;DR, Pinot removed PQL query endpoint and
response format from the current master branch. Only SQL endpoint is supported
starting from 0.11.0 release. More info:  Thanks @jackie.jxt for all the
works!  
 **@darshan.uvce:** @darshan.uvce has joined the channel  

###  _#random_

  
 **@deepakkumar05.it:** @deepakkumar05.it has joined the channel  
 **@darshan.uvce:** @darshan.uvce has joined the channel  

###  _#qps-metric_

  
 **@npawar:** @npawar has left the channel  

###  _#feat-better-schema-evolution_

  
 **@npawar:** @npawar has left the channel  

###  _#troubleshooting_

  
 **@deepakkumar05.it:** @deepakkumar05.it has joined the channel  
 **@darshan.uvce:** @darshan.uvce has joined the channel  

###  _#announcements_

  
 **@darshan.uvce:** @darshan.uvce has joined the channel  

###  _#discuss-validation_

  
 **@npawar:** @npawar has left the channel  

###  _#getting-started_

  
 **@deepakkumar05.it:** @deepakkumar05.it has joined the channel  
 **@darshan.uvce:** @darshan.uvce has joined the channel  

###  _#introductions_

  
 **@deepakkumar05.it:** @deepakkumar05.it has joined the channel  
 **@darshan.uvce:** @darshan.uvce has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org