You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "jiangjiangtian (via GitHub)" <gi...@apache.org> on 2024/09/02 09:34:35 UTC
[I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
jiangjiangtian opened a new issue, #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082
### Backend
VL (Velox)
### Bug description
The SQL as follows will produce wrong result:
```SQL
select value / (value + 0.00001) from table;
```
where value is decimal(20, 0).
The result type needs to be `decimal(38, 19)`. But during calculation, the actual result type is `decimal(38, 20)`:
<img width="125" alt="image" src="https://github.com/user-attachments/assets/1d0b95d8-c93f-4ef9-bb79-193a8e159427">
The reason is that the type of the subexpression `value + 0.00001` will be rescaled to `decimal(27, 5)`:
<img width="202" alt="截屏2024-09-02 17 15 29" src="https://github.com/user-attachments/assets/764f0009-ec13-4fe9-95ef-37e1d4bfb3d5">
But we don't notice it when we rescale the topmost expression:
<img width="206" alt="截屏2024-09-02 17 16 06" src="https://github.com/user-attachments/assets/a73ef178-8499-4ff2-aa66-4dc7d227d3d6">
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2325424530
Hi @jiangjiangtian, would you also post the expected result and incorrect result in this issue? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2324322637
After some investigations, I find that if we have some literals in the decimal arithmetic expressions , we may have wrong result type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2345160337
For now, we pass the result type when we create the decimal function, but it is not an elegant solution, and we will come up with a better solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2325475507
> Hi @jiangjiangtian, would you also post the expected result and incorrect result in this issue? Thanks.
Thanks for reminding me. I add the result in the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2475150011
> You mention matching to the high Spark version, which version would fail now?
Sorry for late reply. To my best knowledge, Spark with version below 3.4 would fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2324276596
@kecookier
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2324282796
Maybe we should not remove the `PromotePrecision(Cast(...))`. Otherwise, we will always have the bug.
Besides, maybe we can also override `DecimalPrecision` in gluten and make it consistent with high version Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Incorrect result type of nested decimal arithmetic expressions [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on issue #7082:
URL: https://github.com/apache/incubator-gluten/issues/7082#issuecomment-2469333032
You mention matching to the high Spark version, which version would fail now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org