Apache Pinot Daily Email Digest (2020-07-20)

[Pinot Slack Email Digest <sn...@apache.org>]

<h3><u>#general</u></h3><br><strong>@ye.chen: </strong>@ye.chen has joined the channel<br><strong>@blcksrx: </strong>@blcksrx has joined the channel<br><strong>@anshuman.bhowmik: </strong>@anshuman.bhowmik has joined the channel<br><strong>@ankit.raj.singh: </strong>@ankit.raj.singh has joined the channel<br><strong>@ankit.raj.singh: </strong>Hi all, can anyone point out what does Ideal state of table means in pinot?...Getting exception: "Ideal state of table tableName does not exist"<br><strong>@mhomaid: </strong><!here> i am currently working on a Kafka connectors (source and sink). The sink is more important I would say. I spent the last two weeks digging into the codebase. I noticed within org.apache.pinot.spi.stream package there are a number of classes/interfaces that are implementing the consumption from Kafka if i am not wrong and within pinot-core -&gt; org.apache.pinot.core.realtime<br><strong>@mhomaid: </strong>I don’t want to reinvent the while here: but my goal is to be able to do the same ( consumer from Kafka using Kafka Connect API -&gt; push/sink the data into pinot<br><strong>@mhomaid: </strong>now i stated my goal<br><strong>@mhomaid: </strong>here is a few screen shots of what i built so far<br><strong>@mhomaid: </strong>i will open source the code this week<br><strong>@mhomaid: </strong><br><strong>@mhomaid: </strong><br><strong>@mhomaid: </strong><br><strong>@mhomaid: </strong><br><strong>@g.kishore: </strong>@mhomaid this is pretty cool, can we do this in batches to start with?<br><strong>@mhomaid: </strong>Yes, that is fine, I just need collaborate with someone in your team at the beginning at least<br><strong>@g.kishore: </strong>@elon.azoulay @afilipchik built something similar to this in Flink to generate the segment and push to Pinot periodically<br><strong>@g.kishore: </strong>lets create a kafka-connect channel and move this discussion to that channel. Super excited about this!<br><strong>@mhomaid: </strong>Awesome<br><h3><u>#random</u></h3><br><strong>@ye.chen: </strong>@ye.chen has joined the channel<br><strong>@blcksrx: </strong>@blcksrx has joined the channel<br><strong>@anshuman.bhowmik: </strong>@anshuman.bhowmik has joined the channel<br><strong>@ankit.raj.singh: </strong>@ankit.raj.singh has joined the channel<br><h3><u>#troubleshooting</u></h3><br><strong>@suraj: </strong>Hello - we are noticing slow queries and wanted to check if there is a way to log the execution plan of the queries in the logs ?<br><strong>@g.kishore: </strong>@suraj you should know what’s taking time by looking at the response stats <br><h3><u>#presto-pinot-streaming</u></h3><br><strong>@jackie.jxt: </strong>You might need to reload files in your IDE<br><strong>@jackie.jxt: </strong>It works for me<br><strong>@jackie.jxt: </strong>@elon.azoulay Reviewed the PR. Please let me know if you need my help wiring the operators together, and I can commit to your PR<br><strong>@elon.azoulay: </strong>Great! Will do, thanks:)<br><strong>@elon.azoulay: </strong>@g.kishore, were you able to build it? I ran mvn clean install -DskipTests -Ppin-dist.<br>