Apache Pinot Daily Email Digest (2020-09-18)

[Pinot Slack Email Digest <sn...@apache.org>]

<h3><u>#general</u></h3><br><strong>@usha: </strong>In this link <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMdUZqJPK6g-2ByLxeLjaNrH-2ByLXOYqM9xBoQCHErAboGEPsoR3KNmsYaowuSMJGCxHoT-2FkPoDZ6LVseift3t8f2Yw-3DyW-Y_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxtsJORoCfG1gt5z9BA-2B4-2FNJlm2TPdYrhgy2S-2Bf-2BXsDuUEuBmJjuxDPv4BLe2btXNApDyigdKTBxMMIyrseO7hbayqeEwql8n2-2FUzpE30TMjgA8FIoBSmG3vjXEaqXIJ-2FYWBXNzPMArKIIRIsE1Hg2dhBcOAy4aTdriLHP9n-2BeDJK6B-2BEyu5crrCAznGGnCeBs-3D>, The author talks about a retention time. Does anyone know if this configurable and where we can set it?<br><strong>@steotia: </strong>Yes it can be set in the table config. See this <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMdTeAXadp8BL3QinSdRtJdp60AJx08J5D0h7qkwrxysZX7HyKSR5zmIogHN68vUWHNkhsPDmRQtZe-2B9ZJtJJCfMr0Qf5CVUiRkr9SlV00-2Bwyg5WavjNw-2FH5-2BBaqqWfZr-2BowXYDVlYQ13wIO4IYeh50m0znRk8px0otHftqx-2BlniXQgyjoBCAvzjzcegKwg883B4Wg-2B-2BFssGbVXl8-2FZbTyxw-3Dly4B_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxtsJORoCfG1gt5z9BA-2B4-2FN1ZrC59jP5M4gPeMl4gWCFX9YXRDZZUDKwxbekJA14xjehkftBglHBpWB-2BYemFQFave72d8OMTD6h-2FdLxHaFaW31dPJUM0a4SN3qRo1PoWBoAjaqZQXYINGtmfbUgIwj2DkEz2C5eNM0BnhLzearBgw-2BAdp93wVAt9eH8kgCBa8E-3D><br><strong>@usha: </strong>Thanks<br><strong>@wrbriggs: </strong>@wrbriggs has joined the channel<br><strong>@wrbriggs: </strong>Hi all! I'm doing some prototyping with Pinot for what seems like a perfect use-case (ingest Avro from Kafka into a single, large, table, and perform fast aggregation and filtering on it, for a rolling window of time). Just to get my feet wet, I've started trying to bootstrap the pinot-quickstart on Kubernetes via Docker Desktop. I've noticed that the `pinot-broker-0` pod is constantly spamming warnings into the log, because there is no zookeeper running:
```2020/09/18 17:48:20.691 WARN [ClientCnxn] [Start a Pinot [BROKER]-SendThread(pinot-zookeeper.pinot-quickstart.svc.cluster.local:2181)] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused```<br><strong>@wrbriggs: </strong>Is this something I should be concerned about, or can I safely ignore it?<br><strong>@mailtobuchi: </strong>A minor feedback on the new QueryConsole UI.
1. The previous version where results are displayed above the response stats was much more intuitive.
2. The response stats printed in JSON was much easier to read and understand than a table. It's a one row table always anyways so I think this should be JSON instead.<br><strong>@shen.wan: </strong>QQ: can we set pinot data explorer query timeout over 25 seconds?<br><strong>@arun.ak37526: </strong>@arun.ak37526 has joined the channel<br><strong>@larrylayup: </strong>@larrylayup has joined the channel<br><strong>@gandhi16.sharayu: </strong>@gandhi16.sharayu has joined the channel<br><h3><u>#random</u></h3><br><strong>@wrbriggs: </strong>@wrbriggs has joined the channel<br><strong>@arun.ak37526: </strong>@arun.ak37526 has joined the channel<br><strong>@larrylayup: </strong>@larrylayup has joined the channel<br><strong>@gandhi16.sharayu: </strong>@gandhi16.sharayu has joined the channel<br><h3><u>#troubleshooting</u></h3><br><strong>@mailtobuchi: </strong>Question: If I add a sort inverted index on a column, what happens to the timestamp column? is the data sorted by the new column only, ignoring the timestamp or is it sorted by both? Not that clear from the docs at <https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMdTeAXadp8BL3QinSdRtJdp7hEkgJimgu1J1h82HsMQqX1-2Fgo-2BaNvnLObBzk-2BXg-2B54ssakBClYDdBqaxbSk4DDHFU288KX7OrQsveYv-2FkV2AdgL1_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTxtsJORoCfG1gt5z9BA-2B4-2FNmEpTdTjzQAVbDo71Cmjy5dnopKnXeaVbk3rtnONTTXIL6nPUXMbpnwWBq2hVPKNudOBJ-2FPA-2B5NEmlLBVQIZ5Lg19mbRXLKGQm-2BoKwrdhmFoj1nrOI427BiBEkAZKVdXcD6eQhTWn7tooFBzA7tNJA6rcmAa4DJun-2F8mbOocHYHk-3D><br><strong>@g.kishore: </strong>what does add sort inverted index mean<br><strong>@mailtobuchi: </strong>let's say i'm adding sort inverted index on `column1`<br><strong>@mayanks: </strong>I think our doc is a bit confusing on `sorted inverted index`.<br><strong>@mayanks: </strong>There's only one indexing file for sorted columns that duals as sorted as well as inverted index.<br><strong>@mayanks: </strong>It is created if the column in input data is sorted.<br><strong>@mayanks: </strong>If you sort your input data on a column, it might (and will likely) unsort the timestamp.<br><strong>@mailtobuchi: </strong>That makes sense. Thanks for clarifying.<br><h3><u>#multiple_streams</u></h3><br><strong>@ssubrama: </strong>@ssubrama has joined the channel<br>