You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "AcKing-Sam (via GitHub)" <gi...@apache.org> on 2024/03/20 04:59:52 UTC
[PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
AcKing-Sam opened a new pull request, #32518:
URL: https://github.com/apache/doris/pull/32518
## Proposed changes
Issue Number: [#31517](https://github.com/apache/doris/issues/31517)
**[Bug] cast date to decimal get wrong result**
Below case got the wrong result:
```mysql
set experimental_enable_nereids_planner = true;
select cast(a as decimal) from (select cast("2000-01-01" as date) a) ta;
mysql> select cast(a as decimal) from (select cast("2000-01-01" as date) a) ta;
+----------------------------+
| cast(a as DECIMALV3(9, 0)) |
+----------------------------+
| 1024033 |
+----------------------------+
1 row in set (0.066 sec)
```
After fix:
```mysql
mysql> select cast(a as decimal) from (select cast("2000-01-01" as date) a) ta;
+----------------------------+
| cast(a as DECIMALV3(9, 0)) |
+----------------------------+
| 20000101 |
+----------------------------+
1 row in set (0.02 sec)
```
## Further comments
For this bug, I read the source code in `be/src/vec/functions/function_cast.h`:
```cpp
if constexpr (IsDataTypeDecimal<FromDataType> || IsDataTypeDecimal<ToDataType>) {
// ...
} else if constexpr (IsTimeType<FromDataType>) {
// ...
} else if constexpr (IsTimeV2Type<FromDataType>) {
// ...
} else {
// ...
}
```
From the if-condition branch, it is clear that `cast from date to decimal` will step into the first branch(from or to decimal), and the under-lying cast for date to uint is missed. This is the reason for this bug.
Therefore, I try to add cast date to int in the first branch.
Then, I read the source code in `be/src/vec/data_types/data_type_decimal.h`. I found the function `convert_to_decimals` is **only for casting between other integral types and decimals**. So I add the cast in `convert_to_decimal` before calling `convert_to_decimals`.
However, I think there must be some potential bugs when casting between date/datetime and decimal, which is a future work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017425595
@zhiqiang-hhhh
Hi, I just tested below in Doris:
```sql
mysql> select cast(c as decimal) from (select cast("9999-12-31" as datetime) c) t3;
ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[E-124]Arithmetic overflow, convert failed from 703674110198153216, expected data is [-999999999, 999999999]
```
For MySQL, the result is:
```sql
+--------------------+
| cast(c as decimal) |
+--------------------+
| 9999999999 |
+--------------------+
```
I think for datetime, current version is okay because of the strict overflow detection in Doris.
I picked this issue from [Bug summary for doris 2.0.4](https://github.com/apache/doris/issues/32234).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017411397
@AcKing-Sam Do we have same issue for Datetime?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032116743
TeamCity be ut coverage result:
Function Coverage: 35.64% (8879/24911)
Line Coverage: 27.37% (72908/266376)
Region Coverage: 26.56% (37708/141993)
Branch Coverage: 23.34% (19214/82314)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3d07e35fe99eee61324d6a8a67b382334b924bcc_3d07e35fe99eee61324d6a8a67b382334b924bcc/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017483265
@zhiqiang-hhhh Got it, I am willing to fix it! :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022324957
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022317175
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #32518:
URL: https://github.com/apache/doris/pull/32518#discussion_r1537571529
##########
regression-test/suites/query_p0/cast/test_cast_from_date_to_decimal.groovy:
##########
@@ -0,0 +1,33 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_cast_from_date_to_decimal") {
+ qt_sql """select cast(a as decimal) from (select cast("2000-01-01" as date) a) t1;"""
+
+ qt_sql """select cast(b as decimal) from (select cast("1000-01-01" as date) b) t2;"""
+
+ qt_sql """select cast(c as decimal) from (select cast("9999-12-31" as date) c) t3;"""
+
+ test {
+ sql """select cast(b as decimal(7, 0)) from (select cast("1000-01-01" as date) b) t2;"""
+ exception "Arithmetic overflow, convert failed from 10000101, expected data is [-9999999, 9999999]"
+ }
+
+ qt_sql """select cast(c as decimal(8, 0)) from (select cast("9999-12-31" as date) c) t3;"""
+
+ qt_sql """select cast(c as decimal(38, 0)) from (select cast("9999-12-31" as date) c) t3;"""
+}
Review Comment:
add some test cases such as:
1. 9999-12-31 23:59:59.999999
2. 0000-00-01
3. 0000-00-00
4. 1999-12-31 13:59:59.999
5. 2000-02-29 11:11:11
6. 2001-02-29 11:11:11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008744597
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008978317
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022446215
TeamCity be ut coverage result:
Function Coverage: 35.25% (8741/24796)
Line Coverage: 27.03% (71552/264701)
Region Coverage: 26.27% (37123/141292)
Branch Coverage: 23.17% (18984/81920)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c900f8a33fa62f06d136ccb2c623510fe2a6294d_c900f8a33fa62f06d136ccb2c623510fe2a6294d/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022288989
@zhiqiang-hhhh
Hi, I wonder what is the correct answer of this sql:
`select cast(a as decimal(30, 4)) from (select cast("9999-12-31 23:59:59.999999" as datetime(6)) a) t1;`
For MYSQL, the answer is:
```sql
+---------------------------+
| cast(a as decimal(30, 4)) |
+---------------------------+
| 99991231235960.0000 |
+---------------------------+
```
For my implementation, it is:
```sql
+-----------------------------+
| cast(a as DECIMALV3(30, 4)) |
+-----------------------------+
| 99991231235959.9999 |
+-----------------------------+
1 row in set (0.02 sec)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034497560
<details>
<summary>TPC-H: <b>Total hot run time: 39177 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d, data reload: false
------ Round 1 ----------------------------------
q1 17592 4111 4104 4104
q2 2015 187 183 183
q3 11133 1309 1463 1309
q4 10693 900 1043 900
q5 8466 3072 3032 3032
q6 219 134 137 134
q7 1151 663 640 640
q8 10116 2058 2086 2058
q9 6816 6358 6210 6210
q10 8429 3578 3510 3510
q11 424 253 238 238
q12 387 226 213 213
q13 17778 2917 2907 2907
q14 274 244 240 240
q15 524 497 484 484
q16 510 397 388 388
q17 971 922 904 904
q18 7632 6595 6443 6443
q19 1622 1563 1544 1544
q20 602 309 307 307
q21 3569 3159 3128 3128
q22 367 315 301 301
Total cold run time: 111290 ms
Total hot run time: 39177 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4060 4053 4069 4053
q2 323 212 217 212
q3 2976 2967 2953 2953
q4 1917 1831 1847 1831
q5 5237 5234 5241 5234
q6 206 124 124 124
q7 2267 1871 1813 1813
q8 3238 3310 3291 3291
q9 8499 8460 8480 8460
q10 3766 3827 3859 3827
q11 565 442 456 442
q12 706 587 563 563
q13 11410 2917 2878 2878
q14 296 261 269 261
q15 513 467 468 467
q16 460 413 418 413
q17 1745 1685 1684 1684
q18 7671 7250 7275 7250
q19 1647 1658 1641 1641
q20 1952 1745 1723 1723
q21 5108 4735 4780 4735
q22 483 420 456 420
Total cold run time: 65045 ms
Total hot run time: 54275 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032142495
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 3d07e35fe99eee61324d6a8a67b382334b924bcc with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 15.5 seconds inserted 10000000 Rows, about 645K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017978812
@AcKing-Sam We should add some test cases for datetime with precision value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2009011009
TeamCity be ut coverage result:
Function Coverage: 35.30% (8730/24730)
Line Coverage: 27.10% (71409/263532)
Region Coverage: 26.34% (37057/140705)
Branch Coverage: 23.25% (18950/81512)
Coverage Report: http://coverage.selectdb-in.cc/coverage/110efc319e8813783f1a77ee28659f728f02b6b8_110efc319e8813783f1a77ee28659f728f02b6b8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2019999160
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2009045106
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2009040132
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2009071193
TeamCity be ut coverage result:
Function Coverage: 35.29% (8728/24730)
Line Coverage: 27.09% (71382/263532)
Region Coverage: 26.33% (37050/140705)
Branch Coverage: 23.23% (18935/81512)
Coverage Report: http://coverage.selectdb-in.cc/coverage/35aad1fba570c4120976fd18e3c97d28ffa0e8bf_35aad1fba570c4120976fd18e3c97d28ffa0e8bf/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016345703
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016366963
TeamCity be ut coverage result:
Function Coverage: 35.26% (8732/24766)
Line Coverage: 27.06% (71510/264305)
Region Coverage: 26.29% (37091/141099)
Branch Coverage: 23.19% (18970/81818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9c41cd32bf7908353989e30a6de8200fee63d281_9c41cd32bf7908353989e30a6de8200fee63d281/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020153769
<details>
<summary>TPC-DS: <b>Total hot run time: 181874 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b90c00c075f5c922e7e2dcacf2ac577dfc30def, data reload: false
query1 949 370 357 357
query2 6551 1913 1918 1913
query3 6702 218 218 218
query4 31689 21315 21367 21315
query5 4333 418 388 388
query6 275 180 188 180
query7 4634 293 288 288
query8 229 179 173 173
query9 9003 2267 2244 2244
query10 569 264 267 264
query11 15696 14353 14226 14226
query12 143 90 90 90
query13 1637 430 439 430
query14 10111 8057 7551 7551
query15 255 200 202 200
query16 8212 273 262 262
query17 1965 577 548 548
query18 2125 293 277 277
query19 351 154 152 152
query20 98 84 92 84
query21 201 124 125 124
query22 4962 4812 4779 4779
query23 33692 32812 32952 32812
query24 10942 2880 2842 2842
query25 594 365 392 365
query26 1200 154 154 154
query27 2641 345 349 345
query28 7257 1891 1886 1886
query29 881 618 599 599
query30 295 147 145 145
query31 987 742 736 736
query32 93 55 61 55
query33 768 253 252 252
query34 1018 480 489 480
query35 833 613 609 609
query36 1023 925 922 922
query37 122 64 65 64
query38 3543 3485 3534 3485
query39 1452 1459 1439 1439
query40 207 116 113 113
query41 50 49 49 49
query42 105 97 97 97
query43 502 455 460 455
query44 1215 755 747 747
query45 283 267 276 267
query46 1118 727 696 696
query47 1914 1840 1844 1840
query48 455 366 363 363
query49 1101 346 336 336
query50 759 375 384 375
query51 6850 6777 6733 6733
query52 103 96 98 96
query53 349 274 278 274
query54 297 234 230 230
query55 86 78 80 78
query56 258 234 225 225
query57 1224 1143 1140 1140
query58 225 198 223 198
query59 2691 2595 2671 2595
query60 259 235 246 235
query61 101 96 95 95
query62 658 453 459 453
query63 303 276 277 276
query64 5751 4012 4066 4012
query65 3087 3030 3024 3024
query66 881 382 380 380
query67 15096 14753 14668 14668
query68 5342 510 512 510
query69 565 378 371 371
query70 1240 1129 1198 1129
query71 435 267 274 267
query72 6763 2845 2670 2670
query73 711 331 327 327
query74 6742 6359 6490 6359
query75 2956 2197 2196 2196
query76 3472 849 845 845
query77 421 268 271 268
query78 11157 10241 10059 10059
query79 4221 526 524 524
query80 1873 388 390 388
query81 566 219 224 219
query82 970 94 92 92
query83 289 159 156 156
query84 291 86 84 84
query85 2001 368 324 324
query86 504 294 312 294
query87 3705 3586 3585 3585
query88 4691 2371 2362 2362
query89 476 371 373 371
query90 2004 179 183 179
query91 171 151 137 137
query92 61 47 50 47
query93 4945 502 483 483
query94 1239 177 182 177
query95 437 334 324 324
query96 611 270 278 270
query97 2669 2471 2460 2460
query98 229 224 214 214
query99 1234 880 897 880
Total cold run time: 296950 ms
Total hot run time: 181874 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022481489
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit c900f8a33fa62f06d136ccb2c623510fe2a6294d with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.7 seconds inserted 10000000 Rows, about 729K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034487301
TeamCity be ut coverage result:
Function Coverage: 35.65% (8882/24915)
Line Coverage: 27.37% (72912/266425)
Region Coverage: 26.54% (37700/142024)
Branch Coverage: 23.35% (19216/82288)
Coverage Report: http://coverage.selectdb-in.cc/coverage/60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d_60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034314616
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032118586
<details>
<summary>TPC-DS: <b>Total hot run time: 180269 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d07e35fe99eee61324d6a8a67b382334b924bcc, data reload: false
query1 1250 382 372 372
query2 6483 1785 1791 1785
query3 6656 228 226 226
query4 24782 21423 21349 21349
query5 4216 418 413 413
query6 276 191 188 188
query7 4604 310 311 310
query8 237 184 194 184
query9 8477 2245 2202 2202
query10 565 262 285 262
query11 14983 14385 14418 14385
query12 144 101 95 95
query13 1648 394 386 386
query14 8624 7146 6778 6778
query15 219 179 191 179
query16 6841 283 293 283
query17 967 612 590 590
query18 1818 292 283 283
query19 205 168 171 168
query20 104 96 96 96
query21 199 127 125 125
query22 4951 4748 4766 4748
query23 33663 32737 32691 32691
query24 12067 3151 3171 3151
query25 696 403 418 403
query26 1906 175 177 175
query27 2952 327 329 327
query28 6785 1839 1808 1808
query29 1139 609 605 605
query30 303 173 171 171
query31 975 736 727 727
query32 100 64 62 62
query33 708 264 273 264
query34 997 499 511 499
query35 840 719 716 716
query36 993 892 907 892
query37 240 81 83 81
query38 3555 3338 3398 3338
query39 1580 1536 1515 1515
query40 296 143 148 143
query41 50 48 48 48
query42 113 105 114 105
query43 445 418 400 400
query44 1107 734 720 720
query45 286 255 265 255
query46 1075 808 762 762
query47 1857 1769 1776 1769
query48 399 310 316 310
query49 1153 369 378 369
query50 803 401 399 399
query51 6895 6687 6722 6687
query52 121 101 99 99
query53 365 299 294 294
query54 329 245 244 244
query55 91 80 77 77
query56 267 248 242 242
query57 1203 1122 1106 1106
query58 259 232 237 232
query59 2381 2272 2351 2272
query60 274 246 265 246
query61 116 113 112 112
query62 710 460 446 446
query63 319 287 293 287
query64 6602 3076 3212 3076
query65 3086 3038 3019 3019
query66 1446 322 318 318
query67 15345 14843 14862 14843
query68 8603 584 602 584
query69 567 347 353 347
query70 1239 1107 1090 1090
query71 493 289 274 274
query72 6303 2592 2435 2435
query73 816 330 330 330
query74 6583 6260 6275 6260
query75 3344 2263 2290 2263
query76 4685 1099 1187 1099
query77 605 257 253 253
query78 11015 10114 10071 10071
query79 10046 537 536 536
query80 1526 438 431 431
query81 520 246 238 238
query82 729 104 110 104
query83 217 169 164 164
query84 273 91 90 90
query85 1501 300 295 295
query86 474 297 285 285
query87 3730 3475 3497 3475
query88 4385 2361 2355 2355
query89 561 377 374 374
query90 1978 175 183 175
query91 139 106 107 106
query92 68 55 56 55
query93 7208 543 537 537
query94 1232 192 202 192
query95 445 335 337 335
query96 615 270 272 270
query97 2676 2486 2503 2486
query98 238 233 212 212
query99 1296 842 836 836
Total cold run time: 299251 ms
Total hot run time: 180269 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016530740
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016548901
<details>
<summary>TPC-H: <b>Total hot run time: 38236 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d29cb284fd6f4258f182d0319f06763c3fb80baf, data reload: false
------ Round 1 ----------------------------------
q1 17628 4604 4195 4195
q2 2109 161 158 158
q3 10596 1186 1222 1186
q4 10229 806 802 802
q5 7471 2984 3007 2984
q6 200 125 123 123
q7 1062 597 576 576
q8 9348 2034 1984 1984
q9 7231 6658 6642 6642
q10 8445 3509 3590 3509
q11 439 224 219 219
q12 401 197 194 194
q13 17795 2837 2862 2837
q14 238 201 219 201
q15 504 477 459 459
q16 509 370 373 370
q17 969 563 616 563
q18 7243 6536 6477 6477
q19 3900 1405 1429 1405
q20 540 255 247 247
q21 3547 2815 2993 2815
q22 348 301 290 290
Total cold run time: 110752 ms
Total hot run time: 38236 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4123 4063 4073 4063
q2 330 234 229 229
q3 2967 2814 2866 2814
q4 1867 1599 1559 1559
q5 5310 5378 5336 5336
q6 202 116 118 116
q7 2256 1854 1844 1844
q8 3173 3303 3305 3303
q9 8728 8725 8716 8716
q10 3785 3826 3784 3784
q11 545 446 450 446
q12 725 528 541 528
q13 16892 2881 2839 2839
q14 284 253 254 253
q15 495 461 457 457
q16 474 425 436 425
q17 1736 1528 1475 1475
q18 7509 7250 7187 7187
q19 1681 1540 1526 1526
q20 1920 1745 1711 1711
q21 4801 4683 4757 4683
q22 535 444 458 444
Total cold run time: 70338 ms
Total hot run time: 53738 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #32518:
URL: https://github.com/apache/doris/pull/32518#discussion_r1536279403
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -655,6 +656,10 @@ void convert_to_decimal(typename ToDataType::FieldType* dst,
for (size_t i = 0; i < size; ++i) {
dst[i].value = FromFieldType(src[i] * multiplier.value + ((src[i] >= 0) ? 0.5 : -0.5));
}
+ } else if constexpr (IsDateV2Type<FromDataType>) {
Review Comment:
please add regression test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020155914
> @yiguolei @zhiqiang-hhhh
>
> Hi, I added the cast from datetime to decimal. Also, I modified the logic in `cast_to_decimals()`. However, there is an exception, to test this SQL: `select cast(a as decimal(30, 9)) from (select cast("9999-12-31 23:59:59.999" as datetime) a) t1;`
>
> In Doris, it directly get the exception: `Conversion from Nullable(DateTimeV2) to Decimal(30, 9) is not supported.`
>
> It looks like the cast from string to datetime cannot make sure the accuracy. Due to this reason, this sql tries to cast a `NULL` to decimal. I think the continuous problem (**accuracy from string to datetime**) can be handled in another issue.
You need to specify the scale of micro-second part.
```sql
select cast("9999-12-31 23:59:59.999" as datetime(3))
--------------
+--------------------------------------------------+
| cast('9999-12-31 23:59:59.999' as DATETIMEV2(3)) |
+--------------------------------------------------+
| 9999-12-31 23:59:59.999 |
+--------------------------------------------------+
1 row in set (0.01 sec)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016530603
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017528076
> @AcKing-Sam btw, maybe we should modify logical in `convert_to_decimals` instead of using a temporary array which costs more memory and does not have performance benefits. For example, add check and do cast to int64 if from type is Date/Datetime.
>
> Do you think which one is better?
Yes, I think the second way is better and I will try to change the committed code.
The code should be like:
```cpp
else if (type is date or datetime) {
// instead of using array: typename FromDataType::FieldType tmp[size];
// directly check here, may reuse some code in convert_to_decimals function
// if okay, directly change to int64
// else take an error
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2019979383
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008666107
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on code in PR #32518:
URL: https://github.com/apache/doris/pull/32518#discussion_r1540928087
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -411,39 +412,86 @@ constexpr bool IsDataTypeDecimalOrNumber =
IsDataTypeDecimal<DataType> || IsDataTypeNumber<DataType>;
// only for casting between other integral types and decimals
+// in addition, this function also handles cast from data/datetime to decimals
template <typename FromDataType, typename ToDataType, bool multiply_may_overflow,
bool narrow_integral, typename RealFrom, typename RealTo>
requires IsDataTypeDecimal<FromDataType> && IsDataTypeDecimal<ToDataType>
void convert_to_decimals(RealTo* dst, const RealFrom* src, UInt32 scale_from, UInt32 scale_to,
const typename ToDataType::FieldType& min_result,
- const typename ToDataType::FieldType& max_result, size_t size) {
+ const typename ToDataType::FieldType& max_result, size_t size,
+ bool is_from_date, bool is_from_datetime) {
Review Comment:
we should check the source data type by deducting from template argument.
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -411,39 +412,86 @@ constexpr bool IsDataTypeDecimalOrNumber =
IsDataTypeDecimal<DataType> || IsDataTypeNumber<DataType>;
// only for casting between other integral types and decimals
+// in addition, this function also handles cast from data/datetime to decimals
template <typename FromDataType, typename ToDataType, bool multiply_may_overflow,
bool narrow_integral, typename RealFrom, typename RealTo>
requires IsDataTypeDecimal<FromDataType> && IsDataTypeDecimal<ToDataType>
Review Comment:
`requires (IsDataTypeDecimal || IsDateTimeV2Type || IsDateV2Type) && IsDataTypeDecimal<ToDataType>`
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -662,7 +710,8 @@ void convert_to_decimal(typename ToDataType::FieldType* dst,
Decimal256, Decimal64>>;
convert_to_decimals<DataTypeDecimal<DecimalFrom>, ToDataType, multiply_may_overflow,
narrow_integral>(dst, src, from_scale, to_scale, min_result, max_result,
- size);
+ size, IsDateV2Type<FromDataType>,
+ IsDateTimeV2Type<FromDataType>);
Review Comment:
IsDateTimeV2Type<FromDataType> should be move to function `convert_to_decimals`, and it will be something like
`convert_to_decimals(...) {
if constexpr (IsDateTimeV2Type<>) {
} else if constexpr (IsDateV2Type) {
} else {
...
}
}
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022324504
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2031997443
@zhiqiang-hhhh Hi, I have added the deduction from template arguments. However, the bot said the function is a read-hard. Could you please give me some hints about how to optimize the code? Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032088447
> @zhiqiang-hhhh Hi, I have added the deduction from template arguments. However, the bot said the function is a read-hard. Could you please give me some hints about how to optimize the code? Thanks a lot!
@AcKing-Sam hello, just leave the bot alone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008984318
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016530928
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016552686
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit d29cb284fd6f4258f182d0319f06763c3fb80baf with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 19.8 seconds inserted 10000000 Rows, about 505K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020159473
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 9b90c00c075f5c922e7e2dcacf2ac577dfc30def with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 22.0 seconds inserted 10000000 Rows, about 454K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020138509
TeamCity be ut coverage result:
Function Coverage: 35.27% (8738/24776)
Line Coverage: 27.04% (71537/264530)
Region Coverage: 26.28% (37109/141221)
Branch Coverage: 23.18% (18979/81882)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9b90c00c075f5c922e7e2dcacf2ac577dfc30def_9b90c00c075f5c922e7e2dcacf2ac577dfc30def/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022437220
<details>
<summary>TPC-H: <b>Total hot run time: 38486 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c900f8a33fa62f06d136ccb2c623510fe2a6294d, data reload: false
------ Round 1 ----------------------------------
q1 17607 4312 4161 4161
q2 2116 159 162 159
q3 10801 1225 1224 1224
q4 10402 753 846 753
q5 7737 3105 3004 3004
q6 210 127 125 125
q7 1099 627 596 596
q8 9724 2120 2044 2044
q9 7303 6788 6679 6679
q10 8410 3415 3596 3415
q11 418 226 225 225
q12 371 204 192 192
q13 17798 2883 2877 2877
q14 263 206 216 206
q15 516 467 465 465
q16 502 380 376 376
q17 964 527 621 527
q18 7216 6631 6528 6528
q19 1558 1482 1508 1482
q20 564 251 257 251
q21 3615 2995 2902 2902
q22 345 302 295 295
Total cold run time: 109539 ms
Total hot run time: 38486 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4116 4198 4056 4056
q2 331 237 229 229
q3 2955 2850 2877 2850
q4 1905 1574 1601 1574
q5 5346 5365 5387 5365
q6 197 117 117 117
q7 2220 1886 1843 1843
q8 3144 3279 3315 3279
q9 8684 8678 8727 8678
q10 3811 3809 3817 3809
q11 533 444 443 443
q12 715 578 529 529
q13 15307 2850 2853 2850
q14 279 243 258 243
q15 493 455 453 453
q16 497 439 419 419
q17 1746 1523 1471 1471
q18 7378 7157 7233 7157
q19 1613 1560 1498 1498
q20 1904 1719 1715 1715
q21 4907 4658 4841 4658
q22 532 452 436 436
Total cold run time: 68613 ms
Total hot run time: 53672 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020035424
@yiguolei @zhiqiang-hhhh
Hi, I added the cast from datetime to decimal. Also, I modified the logic in `cast_to_decimals()`.
However, there is an exception, to test this SQL:
`select cast(a as decimal(30, 9)) from (select cast("9999-12-31 23:59:59.999" as datetime) a) t1;`
In Doris, it directly get the exception:
`Conversion from Nullable(DateTimeV2) to Decimal(30, 9) is not supported.`
It looks like the cast from string to datetime cannot make sure the accuracy. Due to this reason, this sql tries to cast a `NULL` to decimal.
I think the continuous problem (**accuracy from string to datetime**) can be handled in another issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022537980
@zhiqiang-hhhh Got it. Thanks for your review and I will add more regression tests to it. ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032132737
<details>
<summary>ClickBench: <b>Total hot run time: 30.68 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d07e35fe99eee61324d6a8a67b382334b924bcc, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.03
query3 0.24 0.05 0.05
query4 1.68 0.07 0.07
query5 0.50 0.47 0.50
query6 1.12 0.65 0.65
query7 0.02 0.01 0.02
query8 0.05 0.05 0.04
query9 0.57 0.51 0.50
query10 0.56 0.57 0.56
query11 0.15 0.13 0.11
query12 0.14 0.11 0.12
query13 0.61 0.60 0.59
query14 0.77 0.79 0.78
query15 0.87 0.83 0.83
query16 0.35 0.35 0.36
query17 1.01 0.96 0.94
query18 0.25 0.25 0.24
query19 1.90 1.76 1.70
query20 0.01 0.02 0.02
query21 15.41 0.76 0.69
query22 3.15 5.51 2.54
query23 17.59 1.31 1.01
query24 1.96 0.24 0.22
query25 0.15 0.10 0.08
query26 0.30 0.17 0.18
query27 0.09 0.09 0.08
query28 13.61 0.95 0.94
query29 12.68 3.58 3.37
query30 0.25 0.06 0.05
query31 2.87 0.39 0.39
query32 3.27 0.48 0.47
query33 2.83 2.92 2.83
query34 15.53 4.33 4.33
query35 4.38 4.34 4.36
query36 0.67 0.48 0.47
query37 0.22 0.17 0.17
query38 0.15 0.17 0.14
query39 0.05 0.04 0.04
query40 0.18 0.14 0.14
query41 0.09 0.04 0.05
query42 0.06 0.06 0.05
query43 0.05 0.04 0.04
Total cold run time: 106.45 s
Total hot run time: 30.68 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016551923
<details>
<summary>TPC-DS: <b>Total hot run time: 186876 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d29cb284fd6f4258f182d0319f06763c3fb80baf, data reload: false
query1 928 372 342 342
query2 7348 2086 1926 1926
query3 6701 210 218 210
query4 31841 21226 21246 21226
query5 4368 407 396 396
query6 267 185 173 173
query7 4621 296 298 296
query8 223 172 175 172
query9 9245 2280 2260 2260
query10 558 261 252 252
query11 17395 14402 14313 14313
query12 144 92 84 84
query13 1641 434 423 423
query14 12198 11342 11205 11205
query15 310 193 191 191
query16 8225 263 253 253
query17 2004 572 539 539
query18 2095 277 287 277
query19 355 153 155 153
query20 96 86 86 86
query21 200 129 124 124
query22 5013 4906 4802 4802
query23 33175 32622 32811 32622
query24 10658 2926 2889 2889
query25 589 384 388 384
query26 724 156 160 156
query27 2212 364 352 352
query28 6089 1915 1902 1902
query29 849 647 636 636
query30 310 151 150 150
query31 961 739 742 739
query32 98 59 58 58
query33 771 260 340 260
query34 970 476 490 476
query35 831 619 635 619
query36 1039 880 896 880
query37 112 78 82 78
query38 3476 3479 3432 3432
query39 1586 1464 1447 1447
query40 208 118 113 113
query41 51 48 47 47
query42 103 96 93 93
query43 499 445 446 445
query44 1152 758 749 749
query45 271 258 265 258
query46 1123 702 702 702
query47 1919 1831 1876 1831
query48 435 365 366 365
query49 1060 334 337 334
query50 761 380 369 369
query51 6759 6693 6605 6605
query52 117 93 89 89
query53 350 275 273 273
query54 321 253 246 246
query55 84 82 78 78
query56 245 242 245 242
query57 1201 1148 1165 1148
query58 243 210 209 209
query59 2834 2520 2526 2520
query60 268 255 274 255
query61 116 112 113 112
query62 642 473 453 453
query63 304 273 280 273
query64 5429 4084 4101 4084
query65 3077 3038 3042 3038
query66 876 386 374 374
query67 15491 14909 14801 14801
query68 6965 518 526 518
query69 625 395 378 378
query70 1247 1136 1140 1136
query71 524 273 286 273
query72 6372 2712 2538 2538
query73 721 320 323 320
query74 8168 6705 6578 6578
query75 4185 3045 2950 2950
query76 4911 881 951 881
query77 676 280 258 258
query78 10872 10259 10095 10095
query79 9919 536 523 523
query80 1657 400 404 400
query81 533 227 215 215
query82 878 203 209 203
query83 206 145 148 145
query84 286 77 78 77
query85 1355 317 307 307
query86 465 316 287 287
query87 3749 3573 3568 3568
query88 5128 2386 2367 2367
query89 523 373 363 363
query90 1930 176 177 176
query91 185 137 138 137
query92 62 47 47 47
query93 6766 495 488 488
query94 1131 179 174 174
query95 434 338 329 329
query96 608 274 275 274
query97 3055 2896 2882 2882
query98 220 213 211 211
query99 1206 908 889 889
Total cold run time: 311727 ms
Total hot run time: 186876 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020131436
<details>
<summary>TPC-H: <b>Total hot run time: 37996 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b90c00c075f5c922e7e2dcacf2ac577dfc30def, data reload: false
------ Round 1 ----------------------------------
q1 17594 5153 4100 4100
q2 2109 161 158 158
q3 10572 1157 1220 1157
q4 10229 804 879 804
q5 7439 3060 2981 2981
q6 204 120 122 120
q7 1030 591 586 586
q8 9349 2001 1983 1983
q9 7208 6653 6564 6564
q10 8424 3450 3589 3450
q11 434 226 218 218
q12 407 201 199 199
q13 17799 2864 2853 2853
q14 240 207 213 207
q15 524 461 459 459
q16 493 375 373 373
q17 939 531 629 531
q18 7106 6483 6484 6483
q19 1748 1460 1390 1390
q20 559 254 261 254
q21 3600 2940 2831 2831
q22 337 295 308 295
Total cold run time: 108344 ms
Total hot run time: 37996 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4122 4074 4160 4074
q2 336 239 231 231
q3 2999 2829 2854 2829
q4 1831 1575 1534 1534
q5 5351 5374 5364 5364
q6 199 115 118 115
q7 2271 1860 1849 1849
q8 3159 3325 3281 3281
q9 8740 8718 8731 8718
q10 3778 3831 3726 3726
q11 538 445 440 440
q12 735 533 547 533
q13 16924 2850 2833 2833
q14 290 268 256 256
q15 499 447 463 447
q16 473 421 420 420
q17 1732 1487 1462 1462
q18 7435 7159 7126 7126
q19 1776 1573 1524 1524
q20 1936 1721 1725 1721
q21 4830 4765 4698 4698
q22 523 446 455 446
Total cold run time: 70477 ms
Total hot run time: 53627 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022462088
<details>
<summary>TPC-DS: <b>Total hot run time: 181430 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c900f8a33fa62f06d136ccb2c623510fe2a6294d, data reload: false
query1 951 375 361 361
query2 6558 1992 1794 1794
query3 6700 210 214 210
query4 31870 21386 21423 21386
query5 4299 396 391 391
query6 267 195 168 168
query7 4627 296 301 296
query8 233 170 174 170
query9 9230 2281 2265 2265
query10 572 250 243 243
query11 17189 14270 14294 14270
query12 135 97 85 85
query13 1650 419 422 419
query14 10182 7176 7997 7176
query15 260 200 213 200
query16 8242 262 257 257
query17 2004 573 557 557
query18 2121 298 283 283
query19 363 161 159 159
query20 101 91 95 91
query21 204 128 129 128
query22 4989 4863 4829 4829
query23 33405 32906 33009 32906
query24 11970 2844 2891 2844
query25 673 399 386 386
query26 1776 157 158 157
query27 3156 356 354 354
query28 7556 1877 1876 1876
query29 1043 638 632 632
query30 310 154 151 151
query31 960 749 747 747
query32 99 59 55 55
query33 777 265 261 261
query34 1121 490 488 488
query35 839 604 614 604
query36 1015 911 910 910
query37 284 64 69 64
query38 3564 3458 3422 3422
query39 1475 1459 1435 1435
query40 290 117 116 116
query41 53 48 50 48
query42 108 98 96 96
query43 483 438 439 438
query44 1155 744 731 731
query45 287 275 250 250
query46 1103 717 706 706
query47 1922 1858 1864 1858
query48 478 360 354 354
query49 1242 348 339 339
query50 763 369 365 365
query51 6766 6576 6667 6576
query52 113 88 90 88
query53 341 271 282 271
query54 333 251 241 241
query55 87 79 77 77
query56 243 225 232 225
query57 1223 1147 1146 1146
query58 229 215 210 210
query59 2829 2569 2606 2569
query60 267 244 257 244
query61 116 114 116 114
query62 675 462 455 455
query63 305 280 277 277
query64 6911 3973 4026 3973
query65 3095 3027 3061 3027
query66 1439 367 355 355
query67 15384 14955 14960 14955
query68 9045 542 547 542
query69 628 366 378 366
query70 1239 1156 1157 1156
query71 535 259 257 257
query72 6254 2694 2533 2533
query73 1248 312 322 312
query74 8401 6378 6356 6356
query75 4318 2272 2205 2205
query76 5368 902 895 895
query77 805 253 246 246
query78 11039 10196 10087 10087
query79 9739 520 529 520
query80 1219 372 367 367
query81 516 212 219 212
query82 751 91 82 82
query83 201 146 140 140
query84 288 83 78 78
query85 1531 326 310 310
query86 402 313 301 301
query87 3740 3522 3573 3522
query88 4642 2277 2265 2265
query89 515 370 363 363
query90 2032 176 173 173
query91 171 137 135 135
query92 58 48 49 48
query93 6988 504 491 491
query94 1273 173 178 173
query95 446 328 335 328
query96 611 269 269 269
query97 2674 2505 2463 2463
query98 245 219 214 214
query99 1245 887 891 887
Total cold run time: 319177 ms
Total hot run time: 181430 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017407516
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017470658
> @zhiqiang-hhhh Hi, I just tested below in Doris:
>
> ```sql
> mysql> select cast(c as decimal) from (select cast("9999-12-31" as datetime) c) t3;
> ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[E-124]Arithmetic overflow, convert failed from 703674110198153216, expected data is [-999999999, 999999999]
> ```
>
> For MySQL, the result is:
>
> ```sql
> +--------------------+
> | cast(c as decimal) |
> +--------------------+
> | 9999999999 |
> +--------------------+
> ```
>
> I think for datetime, current version is okay because of the strict overflow detection in Doris. I picked this issue from [Bug summary for doris 2.0.4](https://github.com/apache/doris/issues/32234).
This is because we have incorrect integer value of Datetime.
```sql
select cast(c as decimal(30,9)) from (select cast("9999-12-31" as datetime) c) t3
--------------
+------------------------------+
| cast(c as DECIMALV3(30, 9)) |
+------------------------------+
| 703674110198153216.000000000 |
+------------------------------+
1 row in set (0.02 sec)
```
So we need a similar fix for datetime too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on code in PR #32518:
URL: https://github.com/apache/doris/pull/32518#discussion_r1536588367
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -655,6 +656,10 @@ void convert_to_decimal(typename ToDataType::FieldType* dst,
for (size_t i = 0; i < size; ++i) {
dst[i].value = FromFieldType(src[i] * multiplier.value + ((src[i] >= 0) ? 0.5 : -0.5));
}
+ } else if constexpr (IsDateV2Type<FromDataType>) {
+ for (size_t i = 0; i < size; ++i) {
+ dst[i].value = reinterpret_cast<const DateV2Value<DateV2ValueType>&>(src[i]).to_int64();
Review Comment:
1. maybe `assert_cast<const DateV2Value<FromFieldType>&>` is better
2. why convert result to int64? what if target Decimal is Decimal32 or Decimal128?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034520706
<details>
<summary>TPC-DS: <b>Total hot run time: 180581 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d, data reload: false
query1 889 1134 1125 1125
query2 6465 1766 1786 1766
query3 6685 210 213 210
query4 24010 21516 21387 21387
query5 4159 397 405 397
query6 274 189 179 179
query7 4607 296 299 296
query8 239 179 181 179
query9 8479 2218 2223 2218
query10 594 271 268 268
query11 14879 14514 14473 14473
query12 145 100 95 95
query13 1632 391 396 391
query14 8713 6866 6823 6823
query15 214 174 183 174
query16 7064 268 290 268
query17 989 608 573 573
query18 1872 290 278 278
query19 205 163 162 162
query20 101 96 92 92
query21 197 128 128 128
query22 4975 4868 4848 4848
query23 33611 32617 32730 32617
query24 12648 3167 3128 3128
query25 705 403 411 403
query26 1919 169 160 160
query27 3009 332 331 331
query28 6743 1856 1838 1838
query29 1326 596 586 586
query30 312 168 168 168
query31 1000 732 728 728
query32 105 66 59 59
query33 715 265 252 252
query34 1028 474 498 474
query35 887 684 712 684
query36 1017 865 865 865
query37 271 87 81 81
query38 3504 3436 3447 3436
query39 1564 1522 1524 1522
query40 290 136 135 135
query41 49 46 46 46
query42 111 109 104 104
query43 431 388 401 388
query44 1074 707 700 700
query45 271 272 270 270
query46 1081 775 772 772
query47 1910 1830 1808 1808
query48 377 305 314 305
query49 1169 372 360 360
query50 801 395 385 385
query51 6652 6562 6569 6562
query52 114 96 108 96
query53 363 294 295 294
query54 329 248 244 244
query55 92 85 77 77
query56 246 225 234 225
query57 1245 1136 1105 1105
query58 255 221 223 221
query59 2378 2428 2317 2317
query60 245 237 242 237
query61 93 94 88 88
query62 722 456 455 455
query63 310 276 281 276
query64 6372 3058 3097 3058
query65 3051 3026 2996 2996
query66 1475 331 318 318
query67 15571 15255 14825 14825
query68 5075 550 565 550
query69 483 324 324 324
query70 1133 1105 1139 1105
query71 380 279 271 271
query72 6190 2559 2398 2398
query73 708 315 317 315
query74 6823 6249 6446 6249
query75 3001 2306 2289 2289
query76 3086 1004 1206 1004
query77 384 247 245 245
query78 10846 10124 10074 10074
query79 3591 543 525 525
query80 1785 415 418 415
query81 536 245 240 240
query82 1168 113 109 109
query83 269 171 167 167
query84 268 94 98 94
query85 2079 361 332 332
query86 486 275 305 275
query87 3701 3464 3464 3464
query88 4033 2276 2269 2269
query89 506 379 369 369
query90 1957 193 188 188
query91 144 111 112 111
query92 69 53 52 52
query93 5492 534 524 524
query94 1239 198 198 198
query95 442 332 329 329
query96 603 267 273 267
query97 2684 2444 2488 2444
query98 236 213 216 213
query99 1236 853 843 843
Total cold run time: 285461 ms
Total hot run time: 180581 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034540045
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 16.2 seconds inserted 10000000 Rows, about 617K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #32518:
URL: https://github.com/apache/doris/pull/32518#discussion_r1547823646
##########
be/src/vec/data_types/data_type_decimal.h:
##########
@@ -410,40 +411,113 @@ template <typename DataType>
constexpr bool IsDataTypeDecimalOrNumber =
IsDataTypeDecimal<DataType> || IsDataTypeNumber<DataType>;
-// only for casting between other integral types and decimals
+// This function handles cast from data/datetime/integral types to decimals
template <typename FromDataType, typename ToDataType, bool multiply_may_overflow,
bool narrow_integral, typename RealFrom, typename RealTo>
- requires IsDataTypeDecimal<FromDataType> && IsDataTypeDecimal<ToDataType>
+ requires(IsDataTypeDecimal<FromDataType> || IsDateV2Type<FromDataType> ||
+ IsDateTimeV2Type<FromDataType>) &&
+ IsDataTypeDecimal<ToDataType>
void convert_to_decimals(RealTo* dst, const RealFrom* src, UInt32 scale_from, UInt32 scale_to,
Review Comment:
warning: function 'convert_to_decimals' has cognitive complexity of 75 (threshold 50) [readability-function-cognitive-complexity]
```cpp
void convert_to_decimals(RealTo* dst, const RealFrom* src, UInt32 scale_from, UInt32 scale_to,
^
```
<details>
<summary>Additional context</summary>
**be/src/vec/data_types/data_type_decimal.h:433:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if constexpr (IsDateV2Type<FromDataType>) {
^
```
**be/src/vec/data_types/data_type_decimal.h:435:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < size; i++) {
^
```
**be/src/vec/data_types/data_type_decimal.h:436:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if constexpr (multiply_may_overflow) {
^
```
**be/src/vec/data_types/data_type_decimal.h:439:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (common::mul_overflow(static_cast<MaxFieldType>(res).value, multiplier.value,
^
```
**be/src/vec/data_types/data_type_decimal.h:444:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if constexpr (narrow_integral) {
^
```
**be/src/vec/data_types/data_type_decimal.h:445:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (tmp.value < min_result.value || tmp.value > max_result.value) {
^
```
**be/src/vec/data_types/data_type_decimal.h:445:** +1
```cpp
if (tmp.value < min_result.value || tmp.value > max_result.value) {
^
```
**be/src/vec/data_types/data_type_decimal.h:453:** +1, nesting level increased to 3
```cpp
} else {
^
```
**be/src/vec/data_types/data_type_decimal.h:459:** +1, nesting level increased to 1
```cpp
} else if constexpr (IsDateTimeV2Type<FromDataType>) {
^
```
**be/src/vec/data_types/data_type_decimal.h:461:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < size; i++) {
^
```
**be/src/vec/data_types/data_type_decimal.h:465:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if (scale_to < 6) {
^
```
**be/src/vec/data_types/data_type_decimal.h:469:** +1, nesting level increased to 3
```cpp
} else {
^
```
**be/src/vec/data_types/data_type_decimal.h:474:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if constexpr (multiply_may_overflow) {
^
```
**be/src/vec/data_types/data_type_decimal.h:478:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (common::mul_overflow(static_cast<MaxFieldType>(res).value, multiplier.value,
^
```
**be/src/vec/data_types/data_type_decimal.h:482:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if constexpr (narrow_integral) {
^
```
**be/src/vec/data_types/data_type_decimal.h:483:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (tmp.value + datetime_offset < min_result.value ||
^
```
**be/src/vec/data_types/data_type_decimal.h:483:** +1
```cpp
if (tmp.value + datetime_offset < min_result.value ||
^
```
**be/src/vec/data_types/data_type_decimal.h:492:** +1, nesting level increased to 3
```cpp
} else {
^
```
**be/src/vec/data_types/data_type_decimal.h:500:** +1, nesting level increased to 1
```cpp
} else {
^
```
**be/src/vec/data_types/data_type_decimal.h:502:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < size; i++) {
^
```
**be/src/vec/data_types/data_type_decimal.h:503:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if constexpr (multiply_may_overflow) {
^
```
**be/src/vec/data_types/data_type_decimal.h:504:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (common::mul_overflow(static_cast<MaxFieldType>(src[i]).value, multiplier.value,
^
```
**be/src/vec/data_types/data_type_decimal.h:508:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if constexpr (narrow_integral) {
^
```
**be/src/vec/data_types/data_type_decimal.h:509:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (tmp.value < min_result.value || tmp.value > max_result.value) {
^
```
**be/src/vec/data_types/data_type_decimal.h:509:** +1
```cpp
if (tmp.value < min_result.value || tmp.value > max_result.value) {
^
```
**be/src/vec/data_types/data_type_decimal.h:517:** +1, nesting level increased to 3
```cpp
} else {
^
```
**be/src/vec/data_types/data_type_decimal.h:523:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if constexpr (!multiply_may_overflow && narrow_integral) {
^
```
**be/src/vec/data_types/data_type_decimal.h:523:** +1
```cpp
if constexpr (!multiply_may_overflow && narrow_integral) {
^
```
**be/src/vec/data_types/data_type_decimal.h:524:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < size; i++) {
^
```
**be/src/vec/data_types/data_type_decimal.h:525:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if (dst[i].value < min_result.value || dst[i].value > max_result.value) {
^
```
**be/src/vec/data_types/data_type_decimal.h:525:** +1
```cpp
if (dst[i].value < min_result.value || dst[i].value > max_result.value) {
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2032089495
<details>
<summary>TPC-H: <b>Total hot run time: 38861 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d07e35fe99eee61324d6a8a67b382334b924bcc, data reload: false
------ Round 1 ----------------------------------
q1 17619 4216 4099 4099
q2 2017 191 186 186
q3 10530 1207 1396 1207
q4 10216 845 1087 845
q5 7493 2978 2948 2948
q6 222 140 135 135
q7 1128 631 605 605
q8 9419 2054 2077 2054
q9 6667 6263 6186 6186
q10 8411 3516 3511 3511
q11 415 244 249 244
q12 404 223 223 223
q13 17788 2912 2902 2902
q14 272 247 239 239
q15 532 482 495 482
q16 510 379 380 379
q17 959 930 909 909
q18 7273 6529 6457 6457
q19 1632 1547 1551 1547
q20 604 333 299 299
q21 3560 3119 3106 3106
q22 356 298 302 298
Total cold run time: 108027 ms
Total hot run time: 38861 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4070 4053 4024 4024
q2 330 234 217 217
q3 2958 2929 2937 2929
q4 1882 1836 1849 1836
q5 5254 5217 5245 5217
q6 209 121 121 121
q7 2246 1774 1800 1774
q8 3229 3297 3296 3296
q9 8575 8512 8497 8497
q10 3761 4012 4079 4012
q11 543 458 461 458
q12 758 583 606 583
q13 15224 2903 2902 2902
q14 294 272 259 259
q15 509 464 473 464
q16 459 423 402 402
q17 1712 1648 1671 1648
q18 7685 7379 7210 7210
q19 1678 1659 1651 1651
q20 1948 1734 1713 1713
q21 5004 4742 4788 4742
q22 488 429 423 423
Total cold run time: 68816 ms
Total hot run time: 54378 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix cast from date/datetime to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2034533279
<details>
<summary>ClickBench: <b>Total hot run time: 30.11 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 60ffc8e81fdf9ff3d04dd3077cc4ed08e771e63d, data reload: false
query1 0.04 0.03 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.48 0.48 0.50
query6 1.14 0.66 0.66
query7 0.02 0.02 0.01
query8 0.05 0.05 0.04
query9 0.59 0.52 0.51
query10 0.57 0.57 0.57
query11 0.15 0.11 0.10
query12 0.14 0.12 0.12
query13 0.61 0.59 0.59
query14 0.77 0.79 0.80
query15 0.86 0.84 0.82
query16 0.35 0.35 0.36
query17 0.97 1.02 1.00
query18 0.25 0.25 0.24
query19 1.81 1.71 1.70
query20 0.01 0.02 0.01
query21 15.41 0.76 0.74
query22 3.05 6.08 1.59
query23 17.64 1.44 1.01
query24 1.86 0.24 0.22
query25 0.14 0.09 0.08
query26 0.29 0.17 0.18
query27 0.08 0.09 0.09
query28 13.40 0.98 0.93
query29 12.59 3.63 3.50
query30 0.27 0.07 0.06
query31 2.84 0.39 0.38
query32 3.28 0.47 0.47
query33 2.84 2.86 2.86
query34 15.50 4.34 4.34
query35 4.35 4.39 4.36
query36 0.68 0.48 0.47
query37 0.20 0.17 0.18
query38 0.18 0.16 0.16
query39 0.05 0.03 0.04
query40 0.18 0.17 0.16
query41 0.10 0.05 0.06
query42 0.06 0.06 0.05
query43 0.04 0.04 0.05
Total cold run time: 105.81 s
Total hot run time: 30.11 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008662891
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2008710803
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016345912
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016344327
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2016547961
TeamCity be ut coverage result:
Function Coverage: 35.26% (8733/24766)
Line Coverage: 27.05% (71508/264307)
Region Coverage: 26.29% (37101/141096)
Branch Coverage: 23.19% (18971/81816)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d29cb284fd6f4258f182d0319f06763c3fb80baf_d29cb284fd6f4258f182d0319f06763c3fb80baf/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2022521875
> @zhiqiang-hhhh
>
> Hi, I wonder what is the correct answer of this sql: `select cast(a as decimal(30, 4)) from (select cast("9999-12-31 23:59:59.999999" as datetime(6)) a) t1;`
>
> For MYSQL, the answer is:
>
> ```sql
> +---------------------------+
> | cast(a as decimal(30, 4)) |
> +---------------------------+
> | 99991231235960.0000 |
> +---------------------------+
> ```
>
> For my implementation, it is:
>
> ```sql
> +-----------------------------+
> | cast(a as DECIMALV3(30, 4)) |
> +-----------------------------+
> | 99991231235959.9999 |
> +-----------------------------+
> 1 row in set (0.02 sec)
> ```
There is something strange about behaviour of mysql, it is rounding but finished halfway, and doris is just doing truncate.
I just tests pg and sql server ,they all have different behaviour.
I think we can just keep doing truncate, but we need more conner test to make sure that we will do truncate in all situations like above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017983182
@yiguolei Got it, and thanks for the review. I am going to add the cast from datetime to decimal and add the cases. ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2020161566
@AcKing-Sam
```sql
mysql [demo]>select *, cast(str as Datetime(3)) from str;
--------------
select *, cast(str as Datetime(3)) from str
--------------
+------+-------------------------+----------------------------+
| rid | str | cast(str as DATETIMEV2(3)) |
+------+-------------------------+----------------------------+
| 1 | 9999-12-31 23:59:59.999 | 9999-12-31 23:59:59.999 |
+------+-------------------------+----------------------------+
1 row in set (0.09 sec)
mysql [demo]>select *, cast(str as Datetime) from str;
--------------
select *, cast(str as Datetime) from str
--------------
+------+-------------------------+----------------------------+
| rid | str | cast(str as DATETIMEV2(0)) |
+------+-------------------------+----------------------------+
| 1 | 9999-12-31 23:59:59.999 | NULL |
+------+-------------------------+----------------------------+
1 row in set (0.03 sec)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2017509317
@AcKing-Sam btw, maybe we should modify logical in `convert_to_decimals` instead of using a temporary array which costs more memory and does not have performance benefits. For example, add check and do cast to int64 if from type is Date/Datetime.
Do you think which one is better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix] (vectorization) fix_cast_from_date_to_decimal [doris]
Posted by "AcKing-Sam (via GitHub)" <gi...@apache.org>.
AcKing-Sam commented on PR #32518:
URL: https://github.com/apache/doris/pull/32518#issuecomment-2031957277
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org