You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "mneedham (via GitHub)" <gi...@apache.org> on 2023/06/02 11:18:06 UTC
[GitHub] [pinot] mneedham opened a new issue, #10833: Full upsert to partial upsert
mneedham opened a new issue, #10833:
URL: https://github.com/apache/pinot/issues/10833
When using the upsert functionality, it is possible to change the table config from full upsert to partial upsert. But when you make this change, the partial upsert config isn't picked up until you restart the server.
I think this is deliberate as we only create a new TableDataManager if one isn't already designed, otherwise, it uses one that was created for the [first segment added/replaced](https://github.com/apache/pinot/blob/master/pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java#L482) on that server.
To me, this behaviour doesn't seem intuitive. I would expect one of the following to apply:
1. We aren't allowed to change the upsert functionality in this way and table validation should reject our attempts to do so.
2. We should create a new TableDataManager when a new segment is being added/replaced if we detect a change has been made to the table config.
But which of these should it be, or is there another option?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10833: Full upsert to partial upsert
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10833:
URL: https://github.com/apache/pinot/issues/10833#issuecomment-1574379263
Similar to segment reload, we should introduce the table reload operation, which re-create the `TableDataManager` and replace the existing one atomically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] eaugene commented on issue #10833: Full upsert to partial upsert
Posted by "eaugene (via GitHub)" <gi...@apache.org>.
eaugene commented on issue #10833:
URL: https://github.com/apache/pinot/issues/10833#issuecomment-1573573990
Thanks @mneedham for putting it here.
Related discussion: https://apache-pinot.slack.com/archives/C01H1S9J5BJ/p1685437705110729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org