Apache Pinot Daily Email Digest (2021-10-27)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@daniel.bos:** @daniel.bos has joined the channel  
 **@atsushi.sakai:** @atsushi.sakai has joined the channel  
 **@soyinka.majumder:** @soyinka.majumder has joined the channel  
 **@manoj.purohit:** @manoj.purohit has joined the channel  
 **@maximo.alves:** @maximo.alves has joined the channel  
 **@leon.graveland:** @leon.graveland has joined the channel  
 **@pranav.chawla:** @pranav.chawla has joined the channel  
 **@anilkprabhala:** @anilkprabhala has joined the channel  
 **@sandeep.hadoopadmn:** Does Pinot have parcels?, so we can add into
cloudera manager(cdp) and installed?  
 **@diogo.baeder:** Hey guys, just to confirm: this is the official Python
library to access Pinot, correct?  \- I'm asking because of the organization
name mismatch  
**@mayanks:** This is the one folks use  
**@diogo.baeder:** Cool, thanks! :slightly_smiling_face:  
**@diogo.baeder:** So perhaps someone from your team could take a look at
this?  Or would this be handled by other folks?  
**@xiangfu0:** I'll review. Thanks!  
**@diogo.baeder:** Awesome, thanks a lot @xiangfu0! :slightly_smiling_face:  

###  _#random_

  
 **@daniel.bos:** @daniel.bos has joined the channel  
 **@atsushi.sakai:** @atsushi.sakai has joined the channel  
 **@soyinka.majumder:** @soyinka.majumder has joined the channel  
 **@manoj.purohit:** @manoj.purohit has joined the channel  
 **@maximo.alves:** @maximo.alves has joined the channel  
 **@leon.graveland:** @leon.graveland has joined the channel  
 **@pranav.chawla:** @pranav.chawla has joined the channel  
 **@anilkprabhala:** @anilkprabhala has joined the channel  

###  _#troubleshooting_

  
 **@surajkmth29:** Hi Folks, When running the lookup query I see the below
error: ```"message":
"QueryExecutionError:\norg.apache.pinot.spi.exception.BadQueryRequestException:
Caught exception while initializing transform function: lookup\n\tat
org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:207)\n\tat
org.apache.pinot.core.operator.transform.TransformOperator.<init>(TransformOperator.java:56)\n\tat
org.apache.pinot.core.plan.TransformPlanNode.run(TransformPlanNode.java:56)\n\tat
org.apache.pinot.core.plan.SelectionPlanNode.run(SelectionPlanNode.java:83)\n\tat
org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:100)\n\tat
org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33)\n\tat
org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45)\n\tat
org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:296)\n\tat
org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:216)\n\tat
org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60)\n\tat
org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:155)\n\tat
org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:139)\n\tat
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)",
"errorCode": 200 },``` From the error message, it's not very clear to me
what's the issue here. Can anyone help ?  
**@xiangfu0:** whats your query?  
**@surajkmth29:** ```select userId, orgId, lookup('tableB', 'username',
'orgId', orgId, 'userId', userId) as username from tableA limit 10```  
**@xiangfu0:** also is `tableB` a dimension table?  
**@xiangfu0:**  
**@surajkmth29:** Yes tableB is a dimension table  
 **@lrhadoop143:** Hi Folks, How to load orc files in pinot .I used
Classname:org.apache.pinot.plugin.inputformat.orc.ORCRecordReader. but failed
with schema not provided error.  
**@xiangfu0:** can you put the stacktrace  
**@lrhadoop143:**  
**@xiangfu0:** this is the info log, doesn’t seem to be an issue  
**@xiangfu0:** it’s using the schema from the file  
**@lrhadoop143:** Thank you  
 **@moradi.sajjad:** eieecckbhjerbubnbdhjrtilhuritcduvnirujijhfbf  
 **@surajkmth29:** Hi Team, Are aggregation functions supported with lookup
join? I have the below query ```SELECT column1 ,column2 ,lookup('dimTable',
'username', 'orgId', orgId, 'userId', userId) AS username FROM tableA WHERE
column2 IN ('Good') AND column1 != 'Unknown' AND username = 'user' limit
100``` This returns me the result, However, the moment I add either a count(*)
in the select statement or a "group by" the query keeps on loading. Eg:
```SELECT column1 ,column2 ,orgId ,lookup('dimTable', 'username', 'orgId',
orgId, 'userId', userId) AS username FROM tableA WHERE column2 IN ('Good') AND
column1 != 'Unknown' AND username = 'user' GROUP BY column1, column2, orgId,
username limit 100```  
**@mayanks:** What version of Pinot are you using? There was a UI bug where
syntax error in query would not be reported in the UI and it keeps loading  
**@surajkmth29:** The version is 0.8.0.11  
**@surajkmth29:** @mayanks are you referring to  ?  
**@mayanks:** I think this one is probably resolved, can you query broker
directly to eliminate this?  
 **@daniel.bos:** @daniel.bos has joined the channel  
 **@lars-kristian_svenoy:** Hi everyone. I’m looking into making pinot rack
aware when ingesting data from kafka. I can’t find any documentation on this,
is this something that is supported at the moment?  
**@mayanks:** The feature was being worked on at LinkedIn cc: @steotia
@ssubrama  
**@atsushi.sakai:** @atsushi.sakai has joined the channel  
 **@soyinka.majumder:** @soyinka.majumder has joined the channel  
 **@manoj.purohit:** @manoj.purohit has joined the channel  
 **@maximo.alves:** @maximo.alves has joined the channel  
 **@leon.graveland:** @leon.graveland has joined the channel  
 **@pranav.chawla:** @pranav.chawla has joined the channel  
 **@anilkprabhala:** @anilkprabhala has joined the channel  
 **@luisfernandez:** when upgrading pinot do people experience p99 response
time spikes?  
**@mayanks:** It could mean you are operating near capacity, as in if a
replica is unavailable the other replicas see increased load  

### _#pinot-dev_

  
 **@anilkprabhala:** @anilkprabhala has joined the channel  
 **@ruicong.xie:** @ruicong.xie has joined the channel  

###  _#thirdeye-pinot_

  
 **@leon.graveland:** @leon.graveland has joined the channel  
 **@leon.graveland:** Hi everyone, just new to the community. I’m looking into
ThirdEye, and I found that the codebase split from Pinot in order to apply for
Apache Incubator Programme a while ago. Do you guys know where to find the
roadmap for the project?  

###  _#getting-started_

  
 **@manoj.purohit:** @manoj.purohit has joined the channel  
 **@anilkprabhala:** @anilkprabhala has joined the channel  
 **@ruicong.xie:** @ruicong.xie has joined the channel  

###  _#releases_

  
 **@anilkprabhala:** @anilkprabhala has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org