You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zclllyybb (via GitHub)" <gi...@apache.org> on 2025/05/22 09:13:17 UTC
[PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
zclllyybb opened a new pull request, #51163:
URL: https://github.com/apache/doris/pull/51163
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
before:
```sql
select sha2(k0, k1) from str;
```
it works **but the result may doesn't match with the input!**
now:
```sql
ERROR 1105 (HY000): errCode = 2, detailMessage = sha2 needs two params, and the second is must be a integer constant: sha2(k0, k1)
```
### Release note
Fix lost arguments check of function SHA2
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [x] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900490737
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #51163:
URL: https://github.com/apache/doris/pull/51163#discussion_r2102074527
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Sha2.java:
##########
@@ -48,6 +51,22 @@ public Sha2(Expression arg0, Expression arg1) {
super("sha2", arg0, arg1);
}
+ @Override
+ public void checkLegalityBeforeTypeCoercion() {
+ checkLegalityAfterRewrite();
+ }
+
+ @Override
+ public void checkLegalityAfterRewrite() {
+ if ((children.size() != 2) || !(child(1) instanceof IntegerLikeLiteral)) {
+ throw new AnalysisException("sha2 needs two params, and the second is must be a integer constant: " + this.toSql());
+ }
+ final int constParam = ((IntegerLikeLiteral) child(1)).getIntValue();
+ if (!Lists.newArrayList(224, 256, 384, 512).contains(constParam)) {
Review Comment:
please use a static list for Lists.newArrayList(224, 256, 384, 512)
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Sha2.java:
##########
@@ -48,6 +51,22 @@ public Sha2(Expression arg0, Expression arg1) {
super("sha2", arg0, arg1);
}
+ @Override
+ public void checkLegalityBeforeTypeCoercion() {
+ checkLegalityAfterRewrite();
+ }
+
+ @Override
+ public void checkLegalityAfterRewrite() {
+ if ((children.size() != 2) || !(child(1) instanceof IntegerLikeLiteral)) {
+ throw new AnalysisException("sha2 needs two params, and the second is must be a integer constant: " + this.toSql());
+ }
+ final int constParam = ((IntegerLikeLiteral) child(1)).getIntValue();
+ if (!Lists.newArrayList(224, 256, 384, 512).contains(constParam)) {
+ throw new AnalysisException("sha2 functions only support digest length of 224/256/384/512");
Review Comment:
use List to string to avoid exception msg is not same with if condiiton
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Sha2.java:
##########
@@ -48,6 +51,22 @@ public Sha2(Expression arg0, Expression arg1) {
super("sha2", arg0, arg1);
}
+ @Override
+ public void checkLegalityBeforeTypeCoercion() {
+ checkLegalityAfterRewrite();
+ }
+
+ @Override
+ public void checkLegalityAfterRewrite() {
+ if ((children.size() != 2) || !(child(1) instanceof IntegerLikeLiteral)) {
+ throw new AnalysisException("sha2 needs two params, and the second is must be a integer constant: " + this.toSql());
Review Comment:
constant is not same with literal, the log is not reflect the if condition conrrectly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2903462091
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900480507
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900760926
<details>
<summary>TPC-DS: <b>Total hot run time: 186040 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 814100c9dd68eab720159466bfbe2429e2f27e3a, data reload: false
query1 1022 478 482 478
query2 6534 1845 1882 1845
query3 6751 230 224 224
query4 26335 23656 23430 23430
query5 4362 654 449 449
query6 288 200 190 190
query7 4613 485 289 289
query8 288 238 232 232
query9 8628 2612 2610 2610
query10 456 344 264 264
query11 15244 15519 14944 14944
query12 167 118 111 111
query13 1656 529 415 415
query14 8580 6073 6045 6045
query15 214 187 176 176
query16 7092 609 430 430
query17 997 730 576 576
query18 1951 378 287 287
query19 188 190 167 167
query20 118 125 117 117
query21 211 124 103 103
query22 4046 4168 4139 4139
query23 34009 32861 33153 32861
query24 8380 2380 2391 2380
query25 521 464 387 387
query26 1204 277 156 156
query27 2742 495 343 343
query28 4345 2121 2109 2109
query29 698 561 424 424
query30 277 220 192 192
query31 923 856 779 779
query32 72 63 66 63
query33 549 368 320 320
query34 815 840 541 541
query35 790 820 723 723
query36 932 983 915 915
query37 117 101 75 75
query38 4114 4189 4023 4023
query39 1484 1400 1381 1381
query40 213 119 105 105
query41 58 57 56 56
query42 120 114 117 114
query43 516 505 471 471
query44 1310 830 807 807
query45 174 172 170 170
query46 832 1021 635 635
query47 1784 1789 1739 1739
query48 403 415 310 310
query49 795 554 456 456
query50 640 705 433 433
query51 4116 4221 4075 4075
query52 112 101 102 101
query53 223 256 186 186
query54 601 582 537 537
query55 85 83 88 83
query56 309 305 302 302
query57 1128 1130 1083 1083
query58 265 269 254 254
query59 2637 2710 2594 2594
query60 311 322 303 303
query61 128 130 124 124
query62 800 744 649 649
query63 226 186 189 186
query64 4300 1089 781 781
query65 4281 4235 4241 4235
query66 1090 429 326 326
query67 16102 15449 15598 15449
query68 7788 885 522 522
query69 551 326 285 285
query70 1198 1140 1077 1077
query71 428 339 316 316
query72 6138 4746 4951 4746
query73 687 649 346 346
query74 8942 9091 8666 8666
query75 3209 3215 2689 2689
query76 3419 1184 776 776
query77 468 375 399 375
query78 10043 10104 9235 9235
query79 2646 812 593 593
query80 575 494 463 463
query81 498 251 223 223
query82 444 123 96 96
query83 284 256 235 235
query84 304 99 81 81
query85 754 352 358 352
query86 342 334 301 301
query87 4474 4439 4387 4387
query88 3644 2268 2247 2247
query89 404 321 286 286
query90 1924 207 204 204
query91 140 145 109 109
query92 85 65 58 58
query93 2095 925 575 575
query94 726 411 308 308
query95 368 289 281 281
query96 492 581 278 278
query97 2715 2779 2690 2690
query98 230 217 203 203
query99 1411 1404 1280 1280
Total cold run time: 272698 ms
Total hot run time: 186040 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900775149
<details>
<summary>ClickBench: <b>Total hot run time: 29.04 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 814100c9dd68eab720159466bfbe2429e2f27e3a, data reload: false
query1 0.03 0.03 0.04
query2 0.12 0.10 0.10
query3 0.25 0.20 0.20
query4 1.58 0.20 0.20
query5 0.44 0.43 0.45
query6 1.56 0.65 0.66
query7 0.02 0.02 0.02
query8 0.04 0.04 0.04
query9 0.57 0.51 0.52
query10 0.57 0.57 0.57
query11 0.15 0.11 0.11
query12 0.15 0.12 0.11
query13 0.62 0.60 0.60
query14 0.78 0.79 0.81
query15 0.87 0.85 0.85
query16 0.39 0.39 0.38
query17 0.99 1.05 1.04
query18 0.21 0.21 0.21
query19 1.89 1.85 1.82
query20 0.01 0.01 0.01
query21 15.39 0.93 0.55
query22 0.74 1.11 0.66
query23 15.03 1.38 0.63
query24 7.12 1.81 0.82
query25 0.48 0.20 0.13
query26 0.61 0.16 0.13
query27 0.06 0.05 0.04
query28 9.43 0.88 0.45
query29 12.51 3.89 3.26
query30 0.27 0.10 0.07
query31 2.85 0.59 0.40
query32 3.23 0.54 0.46
query33 2.95 3.06 3.05
query34 15.70 5.10 4.43
query35 4.51 4.49 4.48
query36 0.66 0.49 0.49
query37 0.09 0.06 0.06
query38 0.06 0.04 0.04
query39 0.03 0.03 0.02
query40 0.17 0.13 0.12
query41 0.07 0.03 0.03
query42 0.04 0.02 0.03
query43 0.03 0.03 0.03
Total cold run time: 103.27 s
Total hot run time: 29.04 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2903462219
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900728694
<details>
<summary>TPC-H: <b>Total hot run time: 34062 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 814100c9dd68eab720159466bfbe2429e2f27e3a, data reload: false
------ Round 1 ----------------------------------
q1 26036 5275 5025 5025
q2 2078 270 182 182
q3 10400 1248 705 705
q4 10227 1015 552 552
q5 7537 2445 2357 2357
q6 188 166 134 134
q7 945 736 638 638
q8 9327 1318 1209 1209
q9 6965 5050 5219 5050
q10 6881 2304 1879 1879
q11 502 283 276 276
q12 368 363 228 228
q13 17794 3672 3069 3069
q14 236 235 207 207
q15 535 514 494 494
q16 445 434 376 376
q17 604 853 365 365
q18 7592 7193 7023 7023
q19 1212 959 599 599
q20 351 355 230 230
q21 3976 3171 2455 2455
q22 1049 1030 1009 1009
Total cold run time: 115248 ms
Total hot run time: 34062 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5170 5133 5096 5096
q2 237 333 229 229
q3 2144 2676 2327 2327
q4 1380 1811 1366 1366
q5 4604 4459 4383 4383
q6 213 171 130 130
q7 2042 1879 1761 1761
q8 2560 2570 2538 2538
q9 7191 7162 7193 7162
q10 2977 3213 2753 2753
q11 583 522 494 494
q12 686 760 582 582
q13 3556 3917 3346 3346
q14 287 280 267 267
q15 533 490 495 490
q16 455 483 443 443
q17 1171 1532 1394 1394
q18 7784 7628 7337 7337
q19 831 850 912 850
q20 2017 2050 1807 1807
q21 4726 4309 4326 4309
q22 1091 1024 990 990
Total cold run time: 52238 ms
Total hot run time: 50054 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #51163:
URL: https://github.com/apache/doris/pull/51163#discussion_r2106405130
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Sha2.java:
##########
@@ -41,13 +44,31 @@ public class Sha2 extends ScalarFunction
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT).args(VarcharType.SYSTEM_DEFAULT, IntegerType.INSTANCE),
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT).args(StringType.INSTANCE, IntegerType.INSTANCE));
+ private static final List<Integer> validDigest = Lists.newArrayList(224, 256, 384, 512);
Review Comment:
ok. will fix in next pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #51163:
URL: https://github.com/apache/doris/pull/51163
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900618842
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #51163:
URL: https://github.com/apache/doris/pull/51163#issuecomment-2900482277
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](function) Add lost check of function SHA2 in nereids [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #51163:
URL: https://github.com/apache/doris/pull/51163#discussion_r2103934094
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Sha2.java:
##########
@@ -41,13 +44,31 @@ public class Sha2 extends ScalarFunction
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT).args(VarcharType.SYSTEM_DEFAULT, IntegerType.INSTANCE),
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT).args(StringType.INSTANCE, IntegerType.INSTANCE));
+ private static final List<Integer> validDigest = Lists.newArrayList(224, 256, 384, 512);
Review Comment:
static variable's name should use UPPER_CASE name format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org