Apache Pinot Daily Email Digest (2020-07-15)

[Pinot Slack Email Digest <sn...@apache.org>]

<h3><u>#general</u></h3><br><strong>@saileshhps: </strong>@saileshhps has joined the channel<br><strong>@kanth909: </strong>what's a good aws instance for pinot and presto?<br><strong>@shounakmk219: </strong>Hey all, past few months have been very exciting working on pinot. The journey that @gsrisudha03 and I went through and the learnings that we have gained from the pinot community, we have tried to project on the blog. This would not have been possible without the great support that you all provided, especially @g.kishore @fx19880617 @mayanks @ssubrama @kennybastani. Please share it across:grin:
<https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMTptRwTomxdZbZMSGrSstwsCOe9187mCdHQ8wxQK3Hwy555HfXXduR9QVWiFAabLEsHjK68BLRsHbxlnE25RWZmfYzjdwFlT-2FKuo9ZVJngkN2pt4AfUBDqx2sVWAhW7A-2F5IEAzLHaWatdUbSKnu56yuaovQKSsQeGdj1Eka-2FBCSuTV32_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTymj-2FrIP7WWnlD4DwmWkLaPC1Z0WzDkd2owLTJfJWxenAfVjy4EsLHiqZh-2FHKxn2UBlwlAeoSX4SY0wn-2FSdOesrWI7n-2B1PZOnLtmzdnl4OS8uJGqqU36bMpVcrWk-2BVrpV2GFzWIHlB-2FfCLDn4s-2FreNrWNyRHP-2Fkfumyq8-2FSH4MQZMVM1YgDSUy6E5MtKWlb8Ig-3D><br><strong>@g.kishore: </strong>Awesome post @shounakmk219. Thanks for sharing. *Being able to ingest 120k events/sec on a single node is indeed insane!* <br><strong>@ssubrama: </strong>Nice post, @shounakmk219<br><strong>@mayanks: </strong>Great blog as well as great work squeezing insane performance from tiny footprint @shounakmk219 @gsrisudha03 <br><strong>@sleepythread: </strong>@sleepythread has joined the channel<br><strong>@surekha: </strong>@surekha has joined the channel<br><h3><u>#random</u></h3><br><strong>@saileshhps: </strong>@saileshhps has joined the channel<br><strong>@sleepythread: </strong>@sleepythread has joined the channel<br><strong>@surekha: </strong>@surekha has joined the channel<br><h3><u>#pinot-dev</u></h3><br><strong>@yupeng: </strong>hey folks, whats your thought on a rank function like in sql server: <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMY61j4345fI3-2Fqt24kFUCpmUaZIEms-2FyX-2BUVJ202Qyj-2Be-2Fa2K3XAVrc-2FhNjwufX0bo0XpF5LSnmf5JoRpOBh5GL1Wcg2Y3pvTVAc751NhzSbZsAEHZH3w9CJvarLLYPXpw-3D-3D1uS1_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTymj-2FrIP7WWnlD4DwmWkLaPpuxHpZn-2FMK-2FoL0zdYkI7U9h8-2F5Ga-2FYpQP7P50CCBZ6O2G7cUBRXAJg65sw0dVhbPiWDqCtA129hoKgMCZAYpN8YGwSFcbFwBOnSwM38H6zc-2FdLGoUM7tV-2FFXi75CYgRx6Ti8D70zX0580F1KdWHnplIMC-2BkNLRMGt0QwQzaTYro-3D>. I wonder if it can be an intermediate solution to the upsert problem: rank by the timestamp and filter by the top<br><strong>@g.kishore: </strong>will be too expensive<br><strong>@yupeng: </strong>it’s like an agg function?<br><strong>@g.kishore: </strong>no, window functions need partition + sort<br><strong>@g.kishore: </strong>we wont be able to push down the computation to the servers<br><strong>@yupeng: </strong>hmm, i’m thinking of top k here<br><strong>@yupeng: </strong>not a full window function<br><strong>@yupeng: </strong>if it’s top 1, we may have some optimization and turn it into an aggregation problem<br>