You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "gortiz (via GitHub)" <gi...@apache.org> on 2026/02/02 16:50:56 UTC

[PR] Streaming brokers [pinot]

gortiz opened a new pull request, #17613:
URL: https://github.com/apache/pinot/pull/17613

   In SSE, brokers execute the reduce phase, which merges data from different servers. The key point of MSE is to have several map-reduce phases, and each reduce is executed on one or more servers. This means that in MSE, the broker can either receive data from a single server (in which case the data is ready to be sent) or from multiple servers; in the latter case, the final result is guaranteed to be the concatenation of the data from each stage 1 server.
   
   For historical reasons, the broker reads all data sent from stage 1, concatenates it into a Java heap structure, and then serializes it to the format used to be sent to the customer (usually JSON, but recently we also added GRPC). 
   
   This means we need to allocate and retain several copies of the data the customer requests for a while. That may be an issue for queries that return a large amount of data. But there is no technical reason to implement brokers this way. Given the properties described in the first paragraph, brokers could just pipeline the received data to the customer.
   
   This approach reduces the memory cost of running MSE queries in brokers from linear in the number of rows returned to a very low constant. This should also reduce broker latency for large queries. As a side effect, in case of errors, the behavior would be slightly different. Currently, we either return result rows or an error (without rows). In cases where a server finishes successfully while another fails, the result from a streaming broker may include some rows and an error.
   
   As a picture speaks a thousand words, here is a screenshot of a request in master:
   <img width="1710" height="832" alt="image" src="https://github.com/user-attachments/assets/fb39d88c-ceaf-4138-aac1-010c823ef13d" />
   
   And here is the same request in this branch. The query runs a bit faster, but as shown above, the performance may not be that different; the important part is that the customer starts receiving the requests earlier.
   <img width="1710" height="832" alt="image" src="https://github.com/user-attachments/assets/f6dad376-5fc3-4be3-bed2-17d9d6624603" />
   
   This PR also includes a benchmark which executes a simple `select *` query. The difference in raw numbers is not that high because we still need to pay the server-side cost, but the MB/op (`gc.alloc.rate.norm`) metric, especially the total gc time, is significantly better. 
   
   ```
   Benchmark                                                (_segments)  Mode  Cnt           Score      Error   Units
   BenchmarkBrokerSerializer.projectAll                              10  avgt    5         378.497 ±  130.619   ms/op
   BenchmarkBrokerSerializer.projectAll:gc.alloc.rate                10  avgt    5        5918.328 ± 5726.710  MB/sec
   BenchmarkBrokerSerializer.projectAll:gc.alloc.rate.norm           10  avgt    5         2394.66 ±    11.81   MB/op
   BenchmarkBrokerSerializer.projectAll:gc.count                     10  avgt    5          36.000             counts
   BenchmarkBrokerSerializer.projectAll:gc.time                      10  avgt    5         790.000                 ms
   
   Benchmark                                                (_segments)  Mode  Cnt           Score      Error   Units
   BenchmarkBrokerSerializer.projectAll                              10  avgt    5         382.165 ±    7.528   ms/op
   BenchmarkBrokerSerializer.projectAll:gc.alloc.rate                10  avgt    5        6066.623 ± 4853.627  MB/sec
   BenchmarkBrokerSerializer.projectAll:gc.alloc.rate.norm           10  avgt    5         2540.97 ±     11.2   MB/op
   BenchmarkBrokerSerializer.projectAll:gc.count                     10  avgt    5          71.000             counts
   BenchmarkBrokerSerializer.projectAll:gc.time                      10  avgt    5         917.000                 ms
   ```
   
   As reviewers will see, the number of surface changes is pretty large. I was looking for a way to support the old and new modes in the same process, so we can disable this option by default to reduce the risks, but I think that would require even more changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Streaming brokers [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #17613:
URL: https://github.com/apache/pinot/pull/17613#issuecomment-4442705289

   Finally I had time to work on this feature. Now tests seem to be passing. I still want to add an extra change: Only enable this mode when a query option / broker config is used.
   
   Once I have that, it would be ready to merge.
   Given how difficult is to review this PR and how difficult is to resolve conflicts, I would encourage you to start the review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Streaming brokers [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #17613:
URL: https://github.com/apache/pinot/pull/17613#issuecomment-3883558618

   Thanks Ankit! This is something I was trying to implement for a long time, but as you can see, the surface of change is very large. The current code is still not working as expected. I'll try to finish the last mile, but may take a while as I have other higher priority tasks to deal with


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Streaming brokers [pinot]

Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on PR #17613:
URL: https://github.com/apache/pinot/pull/17613#issuecomment-3879763621

   This has been a much needed feature. Thanks @gortiz !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Streaming brokers [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17613:
URL: https://github.com/apache/pinot/pull/17613#issuecomment-3849171480

   ### :x: 3 Tests Failed:
   | Tests completed | Failed | Passed | Skipped |
   |---|---|---|---|
   | 6412 | 3 | 6409 | 0 |
   <details><summary>View the top 3 failed test(s) by shortest run time</summary>
   
   > <pre><code class="language-python">org.apache.pinot.broker.api.resources.PinotClientRequestTest::testPinotQueryComparisonApiDifferentQuery</code></pre>
   > <details><summary>Stack Traces | 0.086s run time</summary>
   > 
   > > <pre><code class="language-python">
   > > Wanted but not invoked:
   > > _requestHandler.handleRequest(
   > >     &lt;Capturing argument: JsonNode&gt;,
   > >     &lt;any&gt;,
   > >     &lt;any&gt;,
   > >     &lt;any&gt;,
   > >     &lt;any&gt;
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequestTest.testPinotQueryComparisonApiDifferentQuery(PinotClientRequestTest.java:165)
   > > 
   > > However, there were exactly 2 interactions with this mock:
   > > _requestHandler.handleStreamingRequest(
   > >     {&quot;sqlV1&quot;:&quot;SELECT v1 FROM mytable&quot;,&quot;sqlV2&quot;:&quot;SELECT v2 FROM mytable&quot;,&quot;sql&quot;:&quot;SELECT v1 FROM mytable&quot;},
   > >     org.apache.pinot.sql.parsers.SqlNodeAndOptions@42c3b02e,
   > >     org.apache.pinot.broker.api.HttpRequesterIdentity@51039ec9,
   > >     org.apache.pinot.spi.trace.DefaultRequestContext@16575bfe,
   > >     null
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequest.executeSqlQuery(PinotClientRequest.java:614)
   > > 
   > > _requestHandler.handleStreamingRequest(
   > >     {&quot;sqlV1&quot;:&quot;SELECT v1 FROM mytable&quot;,&quot;sqlV2&quot;:&quot;SELECT v2 FROM mytable&quot;,&quot;sql&quot;:&quot;SELECT v2 FROM mytable&quot;},
   > >     org.apache.pinot.sql.parsers.SqlNodeAndOptions@60003305,
   > >     org.apache.pinot.broker.api.HttpRequesterIdentity@3f1dadeb,
   > >     org.apache.pinot.spi.trace.DefaultRequestContext@72259917,
   > >     null
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequest.executeSqlQuery(PinotClientRequest.java:614)
   > > 
   > > </code></pre>
   > 
   > </details>
   
   
   > <pre><code class="language-python">org.apache.pinot.broker.api.resources.PinotClientRequestTest::testGetPinotQueryResponse</code></pre>
   > <details><summary>Stack Traces | 0.123s run time</summary>
   > 
   > > <pre><code class="language-python">expected [200] but found [404]</code></pre>
   > 
   > </details>
   
   
   > <pre><code class="language-python">org.apache.pinot.broker.api.resources.PinotClientRequestTest::testPinotQueryComparisonApiSameQuery</code></pre>
   > <details><summary>Stack Traces | 1.01s run time</summary>
   > 
   > > <pre><code class="language-python">
   > > Wanted but not invoked:
   > > _requestHandler.handleRequest(
   > >     &lt;Capturing argument: JsonNode&gt;,
   > >     &lt;Capturing argument: SqlNodeAndOptions&gt;,
   > >     &lt;any&gt;,
   > >     &lt;any&gt;,
   > >     &lt;any&gt;
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequestTest.testPinotQueryComparisonApiSameQuery(PinotClientRequestTest.java:139)
   > > 
   > > However, there were exactly 2 interactions with this mock:
   > > _requestHandler.handleStreamingRequest(
   > >     {&quot;sql&quot;:&quot;SELECT * FROM mytable&quot;},
   > >     org.apache.pinot.sql.parsers.SqlNodeAndOptions@77c4aeb6,
   > >     org.apache.pinot.broker.api.HttpRequesterIdentity@54455c8d,
   > >     org.apache.pinot.spi.trace.DefaultRequestContext@80347b2,
   > >     null
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequest.executeSqlQuery(PinotClientRequest.java:614)
   > > 
   > > _requestHandler.handleStreamingRequest(
   > >     {&quot;sql&quot;:&quot;SELECT * FROM mytable&quot;},
   > >     org.apache.pinot.sql.parsers.SqlNodeAndOptions@eb65fc8,
   > >     org.apache.pinot.broker.api.HttpRequesterIdentity@5cd5e1b6,
   > >     org.apache.pinot.spi.trace.DefaultRequestContext@220a7f95,
   > >     null
   > > );
   > > -&gt; at org.apache.pinot.broker.api.resources.PinotClientRequest.executeSqlQuery(PinotClientRequest.java:614)
   > > 
   > > </code></pre>
   > 
   > </details>
   
   </details>
   
   To view more test analytics, go to the [Test Analytics Dashboard](https://app.codecov.io/gh/apache/pinot/tests/gortiz%3Alazy-brokers)
   <sub>📋 Got 3 mins? [Take this short survey](https://forms.gle/22i53Qa1CySZjA6c7) to help us improve Test Analytics.</sub>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org