You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "deemoliu (via GitHub)" <gi...@apache.org> on 2023/07/25 18:27:10 UTC
[GitHub] [pinot] deemoliu opened a new issue, #11174: [upsert] integrate partial upsert with groovy evaluator
deemoliu opened a new issue, #11174:
URL: https://github.com/apache/pinot/issues/11174
Partial upsert let users choose to update only specific columns and ignore the rest, and apply complex merging logic based on the existing row and the new row that shares the same primary key.
Pinot allows you to run any function using [Apache Groovy](https://groovy-lang.org/) scripts. To support complex use cases (e.g merge previous and new values based on conditions), we proposed to enhance partial upsert with groovy transformation function.
Few challenges to consider
- 1. limitation of Groovy function regex, currently the groovy evaluator doesn't support multi-line groovy script.
- 2. FunctionEvaluator interface currently only take one GenericRow as input.
- 2.1 For partial upsert we need to access both old row and the new row.
- 2.2 How to represent the argument and how to bind the arguments with values for both previous row and the new row.
- 3. Integrate Groovy function in the partial upsert handler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] rohityadav1993 commented on issue #11174: [upsert] integrate partial upsert with groovy evaluator
Posted by "rohityadav1993 (via GitHub)" <gi...@apache.org>.
rohityadav1993 commented on issue #11174:
URL: https://github.com/apache/pinot/issues/11174#issuecomment-1651034368
@Jackie-Jiang, you can assign it to me now. I will share a doc on the approach in the next few days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #11174: [upsert] integrate partial upsert with groovy evaluator
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #11174:
URL: https://github.com/apache/pinot/issues/11174#issuecomment-1650715021
@deemoliu In order to assign an issue to a non-committer, he/she needs to reply to the issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] rohityadav1993 commented on issue #11174: [upsert] integrate partial upsert with groovy evaluator
Posted by "rohityadav1993 (via GitHub)" <gi...@apache.org>.
rohityadav1993 commented on issue #11174:
URL: https://github.com/apache/pinot/issues/11174#issuecomment-1664140082
@deemoliu @Jackie-Jiang , prepared a small doc with initial thought on how to support this: https://docs.google.com/document/d/1bBTCYZFP2stvzc6xZUOEh-XweVgC9WfD7uiSPbKtaZY/edit?usp=sharing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] deemoliu commented on issue #11174: [upsert] integrate partial upsert with groovy evaluator
Posted by "deemoliu (via GitHub)" <gi...@apache.org>.
deemoliu commented on issue #11174:
URL: https://github.com/apache/pinot/issues/11174#issuecomment-1650336153
hello @Jackie-Jiang, this is one of the feature we have discussed with you.
can you assign this task to rohityadav1993 and me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org