You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/08/22 18:04:46 UTC

[GitHub] [pinot] xuhui-stripe opened a new issue, #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

xuhui-stripe opened a new issue, #9266:
URL: https://github.com/apache/pinot/issues/9266

   Hello! We are doing some work to make changes for our tables to utilize "pool based" assignment strategy.
   
   A question that I encountered is do we need to rebalance the table after applying the updated table configs (with pool based configs)? Because we do a fresh ingestion of data to the table daily, I just want to check with you to see if the ingestion that happens daily will pick up the updated table config, and rebalance the data during ingestion? 
   
   The reason I am asking this is, I wonder if it is required to do a rebalance **before** ingesting data to the table that has been updated with the pool based config. (Like do we have to do it before an ingestion happen? Or if a rebalance is required, it could happen at anytime regardless the ingestion.)
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] npawar closed issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
npawar closed issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?
URL: https://github.com/apache/pinot/issues/9266


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9266:
URL: https://github.com/apache/pinot/issues/9266#issuecomment-1224980714

   Yes, you will need to rebalance the table in order to pick up the changes. For segment refresh, without rebalance it will automatically assign the refreshed segment to the same server for the atomic swap.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] xuhui-stripe commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
xuhui-stripe commented on issue #9266:
URL: https://github.com/apache/pinot/issues/9266#issuecomment-1223209677

   Basically the question is shorten to what if a rebalance is not performed then a data ingestion happens for a table~
   Sorry for tagging directly, I feel like this is a quick question~ Thanks for the help. cc @Jackie-Jiang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] xuhui-stripe commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
xuhui-stripe commented on issue #9266:
URL: https://github.com/apache/pinot/issues/9266#issuecomment-1225445547

   Thanks for answering, does it mean it is okay to have for example 1 or 2 segment ingestions/refreshes before actually rebalancing the table? 
   I did a quick test in our QA cluster. If I do an ingestion before rebalancing, the segment-server assignment distribution will not be changed. The distribution will be changed once I hit rebalance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] npawar commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
npawar commented on issue #9266:
URL: https://github.com/apache/pinot/issues/9266#issuecomment-1262940191

   Looks like this has been addressed. Closing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on issue #9266: [Question] whether refreshing/ingesting segments will cause it to rebalance?

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9266:
URL: https://github.com/apache/pinot/issues/9266#issuecomment-1226066957

   That is fine. The ingestion (refresh) won't change the segment assignment, only the rebalance can. Conceptually you can run both of them at the same time (not recommended, but should work)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org