You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2024/06/11 12:59:56 UTC
[PR] Use a two server setup for multi-stage query engine backward compatibility regression test suite [pinot]
yashmayya opened a new pull request, #13371:
URL: https://github.com/apache/pinot/pull/13371
- https://github.com/apache/pinot/pull/13193 added a backward compatibility regression test suite for the multi-stage query engine, leveraging the framework added in https://github.com/apache/pinot/pull/6129, https://github.com/apache/pinot/pull/6650, https://github.com/apache/pinot/issues/4001.
- However, there was one major element missing - the framework relies on a single controller, broker and server setup for Pinot. While this was fine for the v1 query engine backward compatibility tests, using a multi server setup for the multi-stage query engine backward compatibility tests is much more important. Since v2 queries typically involve data shuffling between servers, we also need to test scenarios where one server is running on an older version and another is on a newer version.
- This patch adds an optional second server to the generic framework which is leveraged by the multi-stage query engine backward compatibility regression test suite. Since the segments are added sequentially in the test framework, and the default balanced segment assignment strategy is used (https://docs.pinot.apache.org/operators/operating-pinot/segment-assignment), we know that the queries involving joins between the two tables will result in some data shuffling.
- This isn't easily extensible for adding more controllers / brokers / servers which could've been done via some form of parameterized configuration; however, doing so would likely break existing config file support (https://github.com/apache/pinot/pull/6901) and currently it seems sufficient to have a 1 controller / 1 broker / 2 server setup for testing backward compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Use a two server setup for multi-stage query engine backward compatibility regression test suite [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #13371:
URL: https://github.com/apache/pinot/pull/13371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org