You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/12/14 16:26:47 UTC
[GitHub] [pinot] Airliquide76 opened a new issue, #9989: Derivated column not reloaded when formula changes
Airliquide76 opened a new issue, #9989:
URL: https://github.com/apache/pinot/issues/9989
Derivated column not reloaded when formula changes
Step to reproduce
1- Add a column to the schema save schema
2- Edit table to add a formula like
```
{
"columnName": "actualexpense.amountExcludingTax",
"transformFunction": "\"actualexpense.amountIncludingTax\"-((\"actualexpense.amountIncludingTax\"/(1+(\"actualexpense.tax\"/100)))*(\"actualexpense.tax\"/100))"
},
```
3- Reload Segment
4- Edit table change a formula like
```
{
"columnName": "actualexpense.amountExcludingTax",
"transformFunction": "\"actualexpense.amountIncludingTax\"-((\"actualexpense.amountIncludingTax\"/(1+(\"actualexpense.tax\"/100)))*(\"actualexpense.tax\"/100))-1000"
},
```
5- Reload Segment
Here come my issue my -1000 does not appear in results except in consuming segments. Values should be backfilled as my understanding with the -1000 value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1370286732
Can you try removing the column and reload, then add the updated transform function and reload again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Airliquide76 commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Airliquide76 commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1367390960
Ok many thanks for you feedback. Is there any workaround ? I will have a look (if I've enought time) to try an implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Airliquide76 commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Airliquide76 commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1370616700
I cannot delete colum from schema. Only allowed option add column.
________________________________
De : Xiaotian (Jackie) Jiang ***@***.***>
Envoyé : mercredi 4 janvier 2023 02:04:16
À : apache/pinot
Cc : francois; Author
Objet : Re: [apache/pinot] Derivated column not reloaded when formula changes (Issue #9989)
Which step fails? Can you check if the column index is removed after removing the column from schema and table config?
If the column is not generated on the server side (generated via reload), removing it from schema might not be able to delete its index.
—
Reply to this email directly, view it on GitHub<https://github.com/apache/pinot/issues/9989#issuecomment-1370378478>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/APY2TQ2UFLMHG5WJINS2O4LWQTEBBANCNFSM6AAAAAAS6V4LMM>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1371395434
The force update flag for schema update is introduced in #9382 which will be available in release 12.0 (we are currently in the progress of releasing it). Without that you can try delete the schema first, then re-post it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1362224658
This is a known issue. In order to fix it, we should persist the transform function into the segment metadata, then compare it with the table config to determine whether the column should be re-generated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1370378478
Which step fails? Can you check if the column index is removed after removing the column from **schema** and table config?
If the column is not generated on the server side (generated via reload), removing it from schema might not be able to delete its index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Airliquide76 commented on issue #9989: Derivated column not reloaded when formula changes
Posted by GitBox <gi...@apache.org>.
Airliquide76 commented on issue #9989:
URL: https://github.com/apache/pinot/issues/9989#issuecomment-1370292966
Definitly not working :( already tried
3 janv. 2023 23:25:38 Xiaotian (Jackie) Jiang ***@***.***>:
> Can you try removing the column and reload, then add the updated transform function and reload again?
>
> —
> Reply to this email directly, view it on GitHub[https://github.com/apache/pinot/issues/9989#issuecomment-1370286732], or unsubscribe[https://github.com/notifications/unsubscribe-auth/APY2TQ34DCBEFJI7OXVIH6DWQSRN7ANCNFSM6AAAAAAS6V4LMM].
> You are receiving this because you authored the thread.[Image de pistage][https://github.com/notifications/beacon/APY2TQYG6TIPVKTJRGLKVG3WQSRN7A5CNFSM6AAAAAAS6V4LMOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSRVTVIY.gif]
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org