Apache Pinot Daily Email Digest (2021-12-06)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@krtalic.vedran:** @krtalic.vedran has joined the channel  
 **@diana.arnos:** hey there :wave: I'm not able to make partial upserts work
on 0.9.0 and the error messages are not helpful. the partial upserts did work
previously, I have no idea what is happening. Can someone help me? This kinda
blocks me to deliver my app to prod :sob: More info (such as config files and
error messages) .  
**@mayanks:** @jackie.jxt just fixed an issue in master, so adding him to the
thread  
**@sebastiaan.nicaise:** @sebastiaan.nicaise has joined the channel  

###  _#random_

  
 **@krtalic.vedran:** @krtalic.vedran has joined the channel  
 **@sebastiaan.nicaise:** @sebastiaan.nicaise has joined the channel  

###  _#troubleshooting_

  
 **@krtalic.vedran:** @krtalic.vedran has joined the channel  
 **@sebastiaan.nicaise:** @sebastiaan.nicaise has joined the channel  
 **@nair.a:** Hi Team, We have configured HDFS as deep storage for pinot
cluster. Upon checking the segment.realtime.download.url , it is pointing to
controllers storage path Url:  returned Path:
/tmp/data/PinotController/max_reporting_aggregations/max_reporting_aggregations__0__150__20211127T0641Z
*Shouldn't this segment get stored at HDFS location?* Below are the
controller's and server's config. Controller Config: ```# Pinot Role
pinot.service.role=CONTROLLER # Pinot Cluster name pinot.cluster.name=MAX-
Pinot # Pinot Zookeeper Server pinot.zk.server=c81:2181 # Use hostname as
Pinot Instance ID other than IP pinot.set.instance.id.to.hostname=true # Pinot
Controller Port controller.port=9000 # Pinot Controller VIP Host
controller.vip.host=c81 # Pinot Controller VIP Port controller.vip.port=9000 #
Location to store Pinot Segments pushed from clients controller.data.dir=
controller.task.frequencyPeriod=3600
controller.local.temp.dir=/opt/pinot/host/ controller.enable.split.commit=true
controller.access.protocols.http.port=9000 controller.helix.cluster.name=MAX-
Pinot pinot.controller.segment.fetcher.protocols=file,http,hdfs
pinot.controller.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.controller.storage.factory.hdfs.hadoop.conf.path=/opt/pinot/hadoop/etc/hadoop
pinot.controller.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.grpc.enable=true``` Server Config: ```# Pinot Role
pinot.service.role=SERVER # Pinot Cluster name pinot.cluster.name=MAX-Pinot #
Pinot Zookeeper Server pinot.zk.server=c81:2181 # Use hostname as Pinot
Instance ID other than IP pinot.set.instance.id.to.hostname=true # Pinot
Server Netty Port for queris pinot.server.netty.port=8098 # Pinot Server Admin
API port pinot.server.adminapi.port=8097 # Pinot Server Data Directory
pinot.server.instance.dataDir=/opt/pinot/host/data/server/index # Pinot Server
Temporary Segment Tar Directory
pinot.server.instance.segmentTarDir=/opt/pinot/host/data/server/segmentTar
pinot.server.consumerDir=/opt/pinot/host/data/server/consumer
pinot.server.instance.enable.split.commit=true
pinot.server.instance.reload.consumingSegment=true
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.server.storage.factory.hdfs.hadoop.conf.path=/opt/pinot/hadoop/etc/hadoop
pinot.server.grpc.enable=true pinot.server.grpc.port=8090
pinot.server.query.executor.timeout=100000
pinot.server.instance.realtime.alloc.offheap=true```  
**@mayanks:** The download url should point to HDFS path. Can you check if
PinotFS is initialized to HDFS in controller logs?  
 **@tiger:** Hi, I'm trying to use the Trino/Pinot connector but I'm seeing
some strange behavior. Queries such as `select * from pinot.default.table`
seem to just run forever without returning any results. But queries like
`select max(col) from pinot.default.table` seem to run fine. It looks like I
have to do some sort of aggregation or group by in order for the query to run.
I can't seem to just select rows. Is this behavior expected?  
**@mayanks:** @xiangfu0 ^^  
**@xiangfu0:** have you tried ran with a limit X for your select * query?  
**@tiger:** yep I've tried with a limit  
**@tiger:** As another example: `select col from pinot.default.table limit 10`
seems to run forever without returning. `select col from pinot.default.table
group by col limit 10` works  
**@tiger:** Actually this seems to be fixed after upgrading trino  

###  _#getting-started_

  
 **@michael.bracklo:** @michael.bracklo has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org