Apache Pinot Daily Email Digest (2022-05-19)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@trilokj:** @trilokj has joined the channel  
 **@karinwolok1:** :partying_face: Who wants to come visit in Israel? Submit
to speak at the *FIRST Java Conference in Tel Aviv!*  
**@karinwolok1:** :mega: *Current (formerly Kafka Summit)* is looking for
speakers!!! (Austin, TX in October) If you have a topic that you think might
be valuable to this community, please submit a talk :wink:  
**@karinwolok1:** :mega: *ApacheCon North America* Another open CFP - Closes
on THIS MONDAY, so please submit a talk ASAP! (New Orleans in October)  
**@karinwolok1:** If you get accepted to any conferences, please let us know
and we will send you swag! :smiley:  
 **@deenadhayalan.dd7:** @deenadhayalan.dd7 has joined the channel  
 **@patricio.quintero.c:** @patricio.quintero.c has joined the channel  
 **@matthias.wesolowski:** @matthias.wesolowski has joined the channel  
 **@kis:** operationalizing ml  
**@iyer.vyas:** @iyer.vyas has joined the channel  

###  _#random_

  
 **@trilokj:** @trilokj has joined the channel  
 **@deenadhayalan.dd7:** @deenadhayalan.dd7 has joined the channel  
 **@patricio.quintero.c:** @patricio.quintero.c has joined the channel  
 **@matthias.wesolowski:** @matthias.wesolowski has joined the channel  
 **@iyer.vyas:** @iyer.vyas has joined the channel  

###  _#troubleshooting_

  
 **@mrpringle:** Having some issues with the
RealtimeToOfflineSegmentsTaskExecutor, exception from the logs
`IlleglArgumentExceptionL watermarksMs in RealtimeToOfflineSegmentsTask
metadata {"tableNameWithType":"myTable_REALTIME",
"watermarksMs":16175808000000} does not match windowStartMs: %d in task
configs for table: ....`  
**@mayanks:** @haitao ^^  
**@haitao:** okay, let me take a look  
**@haitao:** this seems like a bug that has already been fixed:  
**@haitao:** it's possible that there is a time gap between different
segments, previously, the gas was not handled properly  
**@mrpringle:** thanks will wait for a release which we can pull in; 0.10 has
vulnerabilities so is blocked here  
**@haitao:** Maybe you can build the jars yourself to unblock yourself for
now?  
 **@mrpringle:** Offline table is not getting populated  
 **@trilokj:** @trilokj has joined the channel  
 **@deenadhayalan.dd7:** @deenadhayalan.dd7 has joined the channel  
 **@abhijeet.kushe:** I am seeing a problem with upsert queries I have all the
primary keys on the record having different values but I am only able to see
those records with `option (skipUpsert=true)` but those records are not
returned with `option (skipUpsert=false)` I added additional primary keys 10
days ago and the segment was created 2 days back.Any ideas what the issue is ?
@npawar  
**@haitao:** did you check this?  
**@haitao:** @kharekartik @jackie.jxt ^  
**@abhijeet.kushe:** I have been using upsert since 7 months have not had the
issue so far.I am using full upsert mode.But thanks for sharing this  
**@haitao:** then it might be a different problem @kharekartik might know more  
**@jackie.jxt:** Do you use time column as the comparison column for upsert?
Is it possible that the segment containing the largest comparison value of the
primary key just expired and removed?  
 **@patricio.quintero.c:** @patricio.quintero.c has joined the channel  
 **@matthias.wesolowski:** @matthias.wesolowski has joined the channel  
 **@iyer.vyas:** @iyer.vyas has joined the channel  

###  _#pinot-dev_

  
 **@francois:** Hi :slightly_smiling_face: Working on integration testing for
the purgeProcess implementation and looking for a way to set `SegmentZkMetada`
my `lastPurgeTime` during the segment build using something like what have
been done in `MergeRollupMinionClusterIntegrationTest` in the
`buildSegmentsFromAvroWithPostfix` method. Is there any way to put a metadata
to the generated segment in order to validate my code ?  

###  _#getting-started_

  
 **@trilokj:** @trilokj has joined the channel  
 **@deenadhayalan.dd7:** @deenadhayalan.dd7 has joined the channel  
 **@patricio.quintero.c:** @patricio.quintero.c has joined the channel  
 **@matthias.wesolowski:** @matthias.wesolowski has joined the channel  
 **@iyer.vyas:** @iyer.vyas has joined the channel  

###  _#introductions_

  
 **@trilokj:** @trilokj has joined the channel  
 **@deenadhayalan.dd7:** @deenadhayalan.dd7 has joined the channel  
 **@patricio.quintero.c:** @patricio.quintero.c has joined the channel  
 **@matthias.wesolowski:** @matthias.wesolowski has joined the channel  
 **@iyer.vyas:** @iyer.vyas has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org