You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "sollhui (via GitHub)" <gi...@apache.org> on 2026/10/22 09:41:37 UTC
[PR] [fix](fe) Avoid logging Kafka Routine Load property values [doris]
sollhui opened a new pull request, #69462:
URL: https://github.com/apache/doris/pull/69462
### What problem does this PR solve?
Issue Number: N/A
Problem Summary:
Altering a Kafka Routine Load job logs the complete data source properties object at INFO level. Its inherited `toString()` includes the original properties map, so Kafka passwords and authentication configuration can appear in FE logs. Edit-log replay uses the same logging path.
Log the job ID and only the changed job/data source property keys. Keep property values intact for runtime configuration and persistence, and preserve support for job-only alterations and absent original property maps.
### Release note
Kafka Routine Load property changes no longer write property values to FE INFO logs.
### Check List (For Author)
- Test
- [x] Unit Test (added; local execution blocked as described below)
- [ ] Regression test
- [ ] Manual test
- [ ] No need to test or manual test.
Added log-capture coverage to `KafkaRoutineLoadJobTest` for normal ALTER and edit-log replay, synthetic SASL/SSL/OAuth values and an ordinary custom value, preserved runtime values, and null/empty data source properties.
Validation:
- Passed: `mvn -o checkstyle:check -pl fe-core` (0 violations).
- Passed: `git diff --check`.
- Attempted: `bash run-fe-ut.sh --run 'org.apache.doris.load.routineload.KafkaRoutineLoadJobTest#testModifyAndReplayPropertiesDoNotLogValues+testModifyJobPropertiesWithoutDataSource'`. The local JDK 17 installation is incomplete (`lib/modules` is missing; `java -version` reports `Failed setting boot class path`), so the script exits during its Java environment check before compiling or executing tests. The unit tests still need CI execution.
- Behavior changed:
- [ ] No.
- [x] Yes. The INFO event retains the job ID and changed property names and omits all property values. Job configuration and persistence are unchanged.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6359811706
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6368512935
<details>
<summary>TPC-H: <b>Total hot run time: 28402 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10e0b398b779048de6272a494db49d711c34463b, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17568 5128 5041 5041
q2 2124 307 250 250
q3 10222 1378 824 824
q4 4681 459 333 333
q5 7436 778 529 529
q6 180 186 156 156
q7 740 784 596 596
q8 9257 1347 1080 1080
q9 5503 4214 4174 4174
q10 6816 1338 1015 1015
q11 428 251 223 223
q12 639 392 279 279
q13 18107 4041 3024 3024
q14 274 273 248 248
q15 q16 717 731 655 655
q17 1205 765 552 552
q18 7039 6218 6127 6127
q19 1099 968 615 615
q20 385 338 231 231
q21 5426 2198 2241 2198
q22 344 300 252 252
Total cold run time: 100190 ms
Total hot run time: 28402 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5613 5617 5584 5584
q2 698 598 554 554
q3 5210 5403 4950 4950
q4 1991 2144 1538 1538
q5 5297 5170 5724 5170
q6 263 193 145 145
q7 2335 2159 1895 1895
q8 3120 2903 2677 2677
q9 8221 8047 8124 8047
q10 3703 3643 3244 3244
q11 601 434 385 385
q12 676 716 519 519
q13 3527 3867 3260 3260
q14 316 305 299 299
q15 q16 696 718 657 657
q17 8201 7586 7210 7210
q18 12945 12184 12878 12184
q19 919 773 767 767
q20 2114 2141 1900 1900
q21 5608 4628 4693 4628
q22 475 444 404 404
Total cold run time: 72529 ms
Total hot run time: 66017 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6382683834
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6350313431
# FE Regression Coverage Report
Increment line coverage ` 41.46% (17/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6351131977
<details>
<summary>TPC-DS: <b>Total hot run time: 150328 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
query5 4308 582 476 476
query6 439 216 191 191
query7 4809 452 223 223
query8 329 158 151 151
query9 8778 3074 3083 3074
query10 465 320 257 257
query11 5974 3890 3551 3551
query12 140 95 86 86
query13 1248 477 316 316
query14 6123 3929 3605 3605
query14_1 3362 3401 3434 3401
query15 220 198 190 190
query16 960 440 407 407
query17 839 647 525 525
query18 2419 424 312 312
query19 205 161 138 138
query20 85 83 82 82
query21 211 127 106 106
query22 12933 12896 12822 12822
query23 13368 12832 12425 12425
query23_1 12544 12614 12460 12460
query24 7394 811 460 460
query24_1 484 489 472 472
query25 542 425 357 357
query26 1253 262 141 141
query27 2780 435 270 270
query28 4639 1969 1930 1930
query29 1555 626 430 430
query30 287 222 181 181
query31 1011 915 803 803
query32 141 95 84 84
query33 507 288 225 225
query34 882 829 485 485
query35 721 742 622 622
query36 838 806 703 703
query37 117 95 80 80
query38 1762 1717 1666 1666
query39 771 784 731 731
query39_1 722 728 734 728
query40 221 110 90 90
query41 64 61 61 61
query42 83 82 80 80
query43 339 369 306 306
query44 1310 682 678 678
query45 176 170 158 158
query46 827 967 576 576
query47 2947 3035 2798 2798
query48 293 306 217 217
query49 585 397 286 286
query50 662 270 214 214
query51 10495 10306 10374 10306
query52 75 79 69 69
query53 184 207 149 149
query54 230 193 172 172
query55 67 71 61 61
query56 226 210 197 197
query57 1631 1633 1598 1598
query58 281 269 251 251
query59 2142 2167 1966 1966
query60 278 248 225 225
query61 143 148 148 148
query62 386 329 295 295
query63 183 155 158 155
query64 2752 924 790 790
query65 3291 3243 3255 3243
query66 1767 411 288 288
query67 20206 19991 19985 19985
query68 2944 1026 605 605
query69 401 308 243 243
query70 842 763 764 763
query71 281 230 206 206
query72 2694 2575 2278 2278
query73 508 512 298 298
query74 4607 4516 4362 4362
query75 2325 2290 1916 1916
query76 2319 1003 596 596
query77 343 382 276 276
query78 9758 9665 9183 9183
query79 1012 850 518 518
query80 1122 473 371 371
query81 577 322 280 280
query82 579 139 108 108
query83 320 202 197 197
query84 317 128 106 106
query85 943 481 362 362
query86 371 237 231 231
query87 1994 2001 1838 1838
query88 3623 2671 2615 2615
query89 329 292 244 244
query90 1815 168 160 160
query91 166 153 122 122
query92 97 85 91 85
query93 986 989 571 571
query94 566 291 260 260
query95 584 401 318 318
query96 655 480 230 230
query97 2463 2444 2321 2321
query98 157 150 147 147
query99 783 757 646 646
Total cold run time: 234331 ms
Total hot run time: 150328 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6350951482
<details>
<summary>TPC-H: <b>Total hot run time: 28402 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17586 5139 5059 5059
q2 2121 330 257 257
q3 10213 1365 820 820
q4 4677 465 339 339
q5 7434 808 533 533
q6 186 191 157 157
q7 742 794 606 606
q8 9238 1364 1046 1046
q9 5454 4209 4175 4175
q10 6831 1284 1042 1042
q11 430 247 234 234
q12 628 401 284 284
q13 18077 3546 2996 2996
q14 266 272 248 248
q15 q16 729 722 667 667
q17 1175 756 575 575
q18 7007 6162 6136 6136
q19 1103 953 592 592
q20 375 332 227 227
q21 5009 2166 2265 2166
q22 346 295 243 243
Total cold run time: 99627 ms
Total hot run time: 28402 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5602 5644 5628 5628
q2 663 551 533 533
q3 5216 5403 4960 4960
q4 2002 2143 1576 1576
q5 5256 5154 5467 5154
q6 252 211 158 158
q7 2289 2008 1833 1833
q8 2998 2629 2629 2629
q9 8016 7903 7970 7903
q10 3778 3701 3227 3227
q11 612 445 582 445
q12 673 708 510 510
q13 3479 3858 3181 3181
q14 325 310 296 296
q15 q16 697 766 656 656
q17 8279 7603 6891 6891
q18 13054 12227 12800 12227
q19 895 768 792 768
q20 2147 2163 1945 1945
q21 5676 4712 4798 4712
q22 490 445 428 428
Total cold run time: 72399 ms
Total hot run time: 65660 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6364971851
<details>
<summary>ClickBench: <b>Total hot run time: 24.23 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4904130dad6ac79752a830c63d9ddab1c78e0480, data reload: false
query1 0.01 0.01 0.01
query2 0.09 0.05 0.05
query3 0.26 0.09 0.09
query4 1.62 0.10 0.11
query5 0.23 0.24 0.25
query6 1.17 0.68 0.68
query7 0.03 0.01 0.01
query8 0.05 0.04 0.03
query9 0.40 0.32 0.35
query10 0.56 0.58 0.57
query11 0.19 0.14 0.15
query12 0.18 0.14 0.14
query13 0.50 0.50 0.49
query14 1.04 1.01 1.01
query15 0.63 0.61 0.60
query16 0.35 0.35 0.35
query17 1.08 1.10 1.15
query18 0.21 0.19 0.19
query19 2.14 1.93 1.96
query20 0.03 0.01 0.01
query21 15.51 0.19 0.15
query22 4.92 0.06 0.06
query23 16.28 0.33 0.12
query24 3.02 0.54 0.37
query25 0.11 0.06 0.05
query26 1.37 0.21 0.15
query27 0.05 0.04 0.04
query28 3.91 0.80 0.35
query29 12.49 4.13 3.25
query30 0.30 0.12 0.09
query31 2.77 0.54 0.32
query32 3.21 0.60 0.49
query33 3.19 3.22 3.24
query34 15.38 4.03 3.40
query35 3.35 3.31 3.34
query36 0.57 0.44 0.44
query37 0.09 0.06 0.06
query38 0.05 0.03 0.03
query39 0.03 0.03 0.03
query40 0.17 0.15 0.15
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.7 s
Total hot run time: 24.23 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6347466918
<details>
<summary>ClickBench: <b>Total hot run time: 24.34 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
query1 0.00 0.00 0.00
query2 0.10 0.05 0.04
query3 0.26 0.09 0.09
query4 1.62 0.11 0.10
query5 0.24 0.24 0.24
query6 1.17 0.68 0.69
query7 0.04 0.01 0.01
query8 0.06 0.04 0.04
query9 0.39 0.36 0.35
query10 0.58 0.61 0.60
query11 0.20 0.14 0.14
query12 0.18 0.15 0.15
query13 0.48 0.50 0.49
query14 1.00 1.01 1.02
query15 0.63 0.64 0.61
query16 0.35 0.34 0.33
query17 1.14 1.16 1.10
query18 0.21 0.19 0.19
query19 2.13 2.03 1.97
query20 0.02 0.01 0.02
query21 15.49 0.19 0.14
query22 4.85 0.06 0.05
query23 16.30 0.31 0.14
query24 3.16 0.48 0.35
query25 0.11 0.06 0.06
query26 0.85 0.21 0.16
query27 0.04 0.03 0.05
query28 3.21 0.79 0.35
query29 12.49 4.07 3.24
query30 0.29 0.13 0.10
query31 2.78 0.54 0.33
query32 3.22 0.58 0.50
query33 3.17 3.19 3.21
query34 15.52 3.96 3.43
query35 3.34 3.32 3.34
query36 0.54 0.48 0.45
query37 0.08 0.06 0.06
query38 0.06 0.04 0.04
query39 0.04 0.03 0.03
query40 0.18 0.15 0.14
query41 0.08 0.03 0.02
query42 0.04 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 96.68 s
Total hot run time: 24.34 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) Avoid logging Kafka Routine Load property values [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6337737943
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6368806872
<details>
<summary>ClickBench: <b>Total hot run time: 24.45 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10e0b398b779048de6272a494db49d711c34463b, data reload: false
query1 0.01 0.00 0.01
query2 0.10 0.05 0.04
query3 0.26 0.10 0.09
query4 1.62 0.10 0.11
query5 0.25 0.26 0.25
query6 1.17 0.70 0.69
query7 0.03 0.01 0.00
query8 0.05 0.04 0.03
query9 0.44 0.36 0.37
query10 0.59 0.62 0.61
query11 0.21 0.15 0.15
query12 0.19 0.15 0.15
query13 0.49 0.50 0.49
query14 1.03 1.02 1.02
query15 0.64 0.62 0.61
query16 0.37 0.36 0.36
query17 1.11 1.14 1.11
query18 0.23 0.22 0.21
query19 2.06 1.92 2.08
query20 0.02 0.02 0.01
query21 15.51 0.20 0.14
query22 4.81 0.06 0.05
query23 16.26 0.33 0.13
query24 2.64 0.52 0.32
query25 0.11 0.07 0.06
query26 1.19 0.21 0.15
query27 0.05 0.03 0.04
query28 3.13 0.72 0.38
query29 12.49 4.21 3.30
query30 0.28 0.14 0.10
query31 2.77 0.58 0.33
query32 3.23 0.62 0.51
query33 3.17 3.23 3.25
query34 15.30 4.20 3.37
query35 3.34 3.34 3.34
query36 0.56 0.46 0.44
query37 0.10 0.06 0.06
query38 0.05 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.15 0.15
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.03 0.04
Total cold run time: 96.23 s
Total hot run time: 24.45 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6360718843
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6353593621
# FE Regression Coverage Report
Increment line coverage ` 41.46% (17/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6357447574
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6363450791
# FE Regression Coverage Report
Increment line coverage ` 8.83% (31/351)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69462:
URL: https://github.com/apache/doris/pull/69462#discussion_r4356893706
##########
fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java:
##########
@@ -701,7 +701,7 @@ public void execute() throws Exception {
TUniqueId queryId = UniqueIdUtils.fastUniqueId();
if (Config.enable_print_request_before_execution) {
LOG.info("begin to execute query {} {}",
- DebugUtil.printId(queryId), getStmtForLoggingBeforeParse());
+ DebugUtil.printId(queryId), getStmtForLogging());
Review Comment:
[P1] Keep startup logs fail-closed for an unsplit batch
With `enable_print_request_before_execution` enabled, submit `SELECT 1; /* outer /* inner */ x; */ ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.password'='synthetic-secret')`. Nereids produces two statements but the nonrecursive separator produces three fragments, so `ConnectProcessor` gives the SELECT the entire request as its `OriginStatement`. Its `UnboundResultSink.needAuditEncryption()` is false, making the new `getStmtForLogging()` return that entire request unchanged and log the password before the ALTER even executes. The changed DEBUG call at line 943 does the same when DEBUG is enabled. Previously both startup sites reparsed the supplied text with `parseSingle` and failed closed on this batch. Ensure the selected plan owns all text being logged, or retain fail-closed masking for an unsplit request, and test real SELECT startup logging in a mismatch batch. This is distinct from thread 4347221252: the ALTER audit event now survives, but another statement's star
tup log newly exposes its secret.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -9422,7 +9427,41 @@ public LogicalPlan visitAlterRoutineLoad(DorisParser.AlterRoutineLoadContext ctx
}
}
- return new AlterRoutineLoadCommand(labelNameInfo, loadPropertyMap, properties, dataSourceMapProperties);
+ return new AlterRoutineLoadCommand(labelNameInfo, loadPropertyMap, properties, dataSourceMapProperties,
+ auditSql);
+ }
+
+ private String buildRoutineLoadAuditSql(DorisParser.AlterRoutineLoadContext ctx) {
+ // Capture masking while the property keys still have the same SQL mode as execution.
+ // Keep only this statement when the request contains multiple statements: the audit caller
+ // can pass the entire request if its separate statement splitter disagrees with Nereids.
+ ParserRuleContext statement = ctx;
+ while (statement.getParent() != null
+ && !(statement.getParent() instanceof MultiStatementsContext)
+ && !(statement.getParent() instanceof SingleStatementContext)) {
+ statement = statement.getParent();
+ }
+ int start = 0;
+ int stop = ctx.start.getInputStream().size() - 1;
+ if (statement.getParent() instanceof MultiStatementsContext
+ && ((MultiStatementsContext) statement.getParent()).statement().size() > 1) {
+ start = statement.start.getStartIndex();
+ stop = statement.stop.getStopIndex();
+ }
+ String sql = ctx.start.getInputStream().getText(new org.antlr.v4.runtime.misc.Interval(start, stop));
+ TreeMap<Pair<Integer, Integer>, String> replacements = new TreeMap<>(new Pair.PairComparator<>());
+ if (ctx.propertyItemList() != null) {
+ for (DorisParser.PropertyItemContext property : ctx.propertyItemList().properties) {
+ if (KafkaPropertiesUtil.isSensitiveKey(parsePropertyKey(property.key))) {
Review Comment:
[P1] Account for the master's mode when capturing follower audit SQL
On a follower connection initially using `NO_BACKSLASH_ESCAPES`, submit `SET sql_mode=''; ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.\password'='synthetic-secret')` for a paused Kafka job. The batch is built before execution, so this classifier sees the literal backslash and retains the secret in `auditSql`. The SET then runs locally; ALTER is forwarded before validation, and `FEOpExecutor` sends its original SQL with the current default SQL mode. The master reparses it as the valid `property.sasl.password` and applies the credential, while the follower keeps its original command and `AuditLogHelper` logs that command's unmasked capture. Its optional startup logs also use that capture. Account for every interpretation that can execute as a credential key, or preserve the execution parse mode across forwarding, and add a real follower/master batch regression. Unlike thread 4347221246's repaired direct-master audit-time reparse, this path reparses for execution on anot
her FE, so the local initial key is not the executed key.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6363871385
# FE Regression Coverage Report
Increment line coverage ` 33.33% (31/93)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6368709357
<details>
<summary>TPC-DS: <b>Total hot run time: 150442 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10e0b398b779048de6272a494db49d711c34463b, data reload: false
query5 4327 593 448 448
query6 435 203 188 188
query7 4811 440 224 224
query8 321 166 157 157
query9 8798 3064 3107 3064
query10 489 302 261 261
query11 5913 3848 3551 3551
query12 151 95 92 92
query13 1268 449 327 327
query14 6095 3938 3635 3635
query14_1 3386 3437 3395 3395
query15 206 201 180 180
query16 942 453 354 354
query17 888 658 538 538
query18 2412 427 308 308
query19 206 166 138 138
query20 87 85 83 83
query21 211 129 106 106
query22 12990 13055 12929 12929
query23 13649 13011 12501 12501
query23_1 12538 12510 12661 12510
query24 7383 807 469 469
query24_1 480 479 462 462
query25 538 418 361 361
query26 1277 251 137 137
query27 2799 439 274 274
query28 4621 1927 1938 1927
query29 1581 600 461 461
query30 310 224 195 195
query31 1007 937 830 830
query32 151 99 88 88
query33 527 301 249 249
query34 917 851 500 500
query35 742 747 648 648
query36 813 811 707 707
query37 125 97 81 81
query38 1768 1739 1661 1661
query39 801 791 761 761
query39_1 737 729 729 729
query40 223 118 98 98
query41 72 67 68 67
query42 89 84 84 84
query43 346 358 313 313
query44 1303 684 680 680
query45 183 187 168 168
query46 818 991 574 574
query47 3031 2986 2903 2903
query48 280 302 222 222
query49 567 406 287 287
query50 674 270 207 207
query51 10564 10688 10444 10444
query52 78 78 72 72
query53 182 210 154 154
query54 255 192 173 173
query55 75 66 62 62
query56 214 203 203 203
query57 1642 1602 1561 1561
query58 282 258 264 258
query59 2123 2123 1921 1921
query60 279 241 224 224
query61 144 147 141 141
query62 397 325 287 287
query63 181 157 157 157
query64 2711 954 731 731
query65 3311 3254 3269 3254
query66 1782 409 299 299
query67 20081 19912 19917 19912
query68 3009 982 596 596
query69 388 286 249 249
query70 820 749 747 747
query71 299 239 215 215
query72 2752 2612 2038 2038
query73 502 521 293 293
query74 4630 4499 4279 4279
query75 2323 2288 1910 1910
query76 2314 1009 577 577
query77 355 352 273 273
query78 9851 9775 9272 9272
query79 986 838 518 518
query80 638 413 349 349
query81 553 308 277 277
query82 573 128 101 101
query83 268 199 181 181
query84 309 126 100 100
query85 892 422 372 372
query86 346 244 228 228
query87 1971 1952 1789 1789
query88 3634 2657 2640 2640
query89 339 281 241 241
query90 1803 168 167 167
query91 162 150 126 126
query92 99 88 88 88
query93 993 983 576 576
query94 473 303 263 263
query95 571 387 297 297
query96 672 487 229 229
query97 2445 2445 2342 2342
query98 159 159 145 145
query99 776 767 652 652
Total cold run time: 234247 ms
Total hot run time: 150442 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6338163797
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6352610294
# FE Regression Coverage Report
Increment line coverage ` 41.46% (17/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6365720849
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) Avoid logging Kafka Routine Load property values [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6337736251
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69462:
URL: https://github.com/apache/doris/pull/69462#discussion_r4356179972
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/kafka/KafkaPropertiesUtil.java:
##########
@@ -0,0 +1,70 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.load.routineload.kafka;
+
+import org.apache.doris.common.util.DatasourcePrintableMap;
+
+import org.apache.commons.collections4.MapUtils;
+
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+
+/** Shared masking for Kafka configuration displays, modification logs and audit SQL. */
+public final class KafkaPropertiesUtil {
+ public static final String SENSITIVE_PROPERTY_MASK = "******";
+
+ private KafkaPropertiesUtil() {
+ }
+
+ public static Map<String, String> maskProperties(Map<String, String> properties) {
+ Map<String, String> maskedProperties = new HashMap<>();
+ MapUtils.emptyIfNull(properties).forEach((key, value) -> maskedProperties.put(key,
+ isSensitiveKey(key) ? SENSITIVE_PROPERTY_MASK : value));
+ return maskedProperties;
+ }
+
+ public static boolean isSensitiveKey(String key) {
+ String lowerKey = key.toLowerCase(Locale.ROOT);
Review Comment:
[P2] Classify Kafka credential keys at their native NUL boundary
In default SQL mode, `ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.password\0ignored'='synthetic-secret')` decodes the key to an interior NUL and passes the prefix-only Kafka property validation. This classifier sees `sasl.password<NUL>ignored`, matches no sensitive rule, and leaves the credential visible in both the new audit SQL and modification/replay INFO log. The unmodified key reaches BE `KafkaDataConsumer::init`; the pinned [librdkafka v2.16.0 setter](https://github.com/confluentinc/librdkafka/blob/v2.16.0/src-cpp/ConfImpl.cpp#L34-L44) passes `name.c_str()` to its C API, so the effective property is `sasl.password`, not an unsupported key. Classify that effective name, or conservatively mask NUL-containing keys, and cover this alias in audit and INFO-log tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6360727890
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6365711682
# FE Regression Coverage Report
Increment line coverage ` 33.33% (31/93)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69462:
URL: https://github.com/apache/doris/pull/69462#discussion_r4360080629
##########
fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java:
##########
@@ -2535,8 +2536,16 @@ private String getStmtForLogging(String stmt) {
}
}
- private String getStmtForLoggingBeforeParse() {
- return getStmtForLoggingBeforeParse(originStmt == null ? null : originStmt.originStmt);
+ private String getStmtForLogging() {
+ String stmt = originStmt == null ? null : originStmt.originStmt;
+ if (parsedStmt instanceof LogicalPlanAdapter
+ && ((LogicalPlanAdapter) parsedStmt).getLogicalPlan() instanceof AlterRoutineLoadCommand) {
+ // This command captures only its own masked SQL during the initial parse.
+ return getStmtForLogging(stmt);
Review Comment:
[P1] Verify statement ownership before trusting captured ALTER SQL
On a follower initially using `NO_BACKSLASH_ESCAPES`, submit this batch for an existing paused Kafka job:
```sql
SET sql_mode='';
SELECT "prefix\";
ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.client.id'="; ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.password'='synthetic-secret') -- ");
```
Nereids and `SqlUtils.splitMultiStmts` both return three statements, but the separator always interprets backslash escapes. Nereids's third command has only `property.client.id`, with the credential ALTER inside its double-quoted value; the separator's third fragment is that credential ALTER itself, with the remaining text commented out. `ConnectProcessor` checks only equal counts and attaches the latter fragment as this command's `OriginStatement`. After SET clears the mode, `FEOpExecutor` forwards that fragment and the master applies `sasl.password='synthetic-secret'`.
This new fast path instead logs the original command's frozen capture, and follower audit uses the same capture. Its isolated ALTER slice has no backslash and only a nonsensitive client.id key, so both INFO/DEBUG startup output and the follower audit event retain the actual password. Establish capture/OriginStatement ownership, or check mode-dependent boundaries over the full request before slicing and fail closed. Add an equal-count/different-boundary follower regression. Unlike threads 4356893714 and 4356893706, the current slice is identical under both modes and the statement counts agree; the boundary change is in the preceding SELECT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6338165297
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6358962052
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6352968596
# FE Regression Coverage Report
Increment line coverage ` 41.46% (17/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6358968867
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6347180089
<details>
<summary>TPC-H: <b>Total hot run time: 28446 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17603 5148 5047 5047
q2 2082 298 260 260
q3 10263 1355 823 823
q4 4679 457 335 335
q5 7422 791 542 542
q6 178 183 151 151
q7 740 818 602 602
q8 9251 1289 1086 1086
q9 5409 4214 4194 4194
q10 6836 1296 1039 1039
q11 420 246 228 228
q12 631 392 278 278
q13 18114 3558 3005 3005
q14 267 268 244 244
q15 q16 725 713 666 666
q17 1181 741 567 567
q18 6941 6150 6112 6112
q19 1106 950 601 601
q20 374 328 221 221
q21 5035 2175 2308 2175
q22 348 300 270 270
Total cold run time: 99605 ms
Total hot run time: 28446 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5628 5610 5613 5610
q2 657 544 551 544
q3 5189 5365 4940 4940
q4 1998 2195 1541 1541
q5 5224 5126 5420 5126
q6 264 210 169 169
q7 2234 2127 1866 1866
q8 3015 2668 2644 2644
q9 7955 8100 8164 8100
q10 3772 3717 3351 3351
q11 619 449 419 419
q12 855 757 535 535
q13 3596 3931 3224 3224
q14 322 320 291 291
q15 q16 727 722 645 645
q17 8278 7540 6896 6896
q18 12999 12138 12864 12138
q19 914 827 818 818
q20 2143 2163 1925 1925
q21 5618 4577 4774 4577
q22 484 448 402 402
Total cold run time: 72491 ms
Total hot run time: 65761 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6364854431
# FE UT Coverage Report
Increment line coverage ` 87.10% (81/93)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_4904130dad6ac79752a830c63d9ddab1c78e0480/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6369635343
# FE Regression Coverage Report
Increment line coverage ` 32.98% (31/94)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_10e0b398b779048de6272a494db49d711c34463b_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_10e0b398b779048de6272a494db49d711c34463b_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6384467984
# FE Regression Coverage Report
Increment line coverage ` 35.05% (34/97)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_3eb671efdd667136aea10b75b7a229fe8c820a1d_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_3eb671efdd667136aea10b75b7a229fe8c820a1d_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69462:
URL: https://github.com/apache/doris/pull/69462#discussion_r4347221246
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilderForEncryption.java:
##########
@@ -104,6 +105,20 @@ public SetVarOp visitSetLdapAdminPassword(DorisParser.SetLdapAdminPasswordContex
return super.visitSetLdapAdminPassword(ctx);
}
+ @Override
+ public LogicalPlan visitAlterRoutineLoad(DorisParser.AlterRoutineLoadContext ctx) {
+ if (ctx.propertyItemList() != null) {
+ for (DorisParser.PropertyItemContext property : ctx.propertyItemList().properties) {
+ if (KafkaPropertiesUtil.isSensitiveKey(parsePropertyKey(property.key))) {
Review Comment:
[P1] Decode audit keys using the original statement's SQL mode
On a master FE, submit one batch under the default escape mode: `SET sql_mode='NO_BACKSLASH_ESCAPES'; ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.\password' = 'synthetic-secret')`. `ConnectProcessor` parses the whole batch before execution, so the ALTER command already contains the valid key `property.sasl.password` and applies it to the job. After the SET executes, this audit-time `parsePropertyKey` instead preserves the backslash; the classifier misses `sasl.\password` and leaves the credential plaintext in audit/statement SQL. The text is valid in both modes, so the logging fallback does not help. Preserve the original parse mode or the original decoded key/range information for masking, and add a batch regression that changes SQL mode before ALTER.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterRoutineLoadCommand.java:
##########
@@ -353,4 +353,9 @@ public <R, C> R accept(PlanVisitor<R, C> visitor, C context) {
public StmtType stmtType() {
return StmtType.ALTER;
}
+
+ @Override
+ public boolean needAuditEncryption() {
+ return true;
Review Comment:
[P2] Preserve ALTER audit events when statement splitting disagrees
The new encryption hook assumes its SQL is a single statement, but `ConnectProcessor` passes the entire request when `SqlUtils.splitMultiStmts` returns a different count from Nereids. A valid default-mode example is `ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.password'='secret'); /* outer /* inner */ x; */ SELECT 1`: Nereids handles the nested comment and builds two commands, while the separator closes the comment at the inner `*/` and retains three nonempty fragments. The ALTER still runs, but its new `geneEncryptionSQL` call reparses the full request with `singleStatement` and throws; `AuditLogHelper.logAuditLog` catches this before submitting any event, so the ALTER disappears from audit logs. Ensure masking receives the original statement's correct range, or safely handles this fallback, and cover the real multi-statement audit path.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilderForEncryption.java:
##########
@@ -104,6 +105,20 @@ public SetVarOp visitSetLdapAdminPassword(DorisParser.SetLdapAdminPasswordContex
return super.visitSetLdapAdminPassword(ctx);
}
+ @Override
+ public LogicalPlan visitAlterRoutineLoad(DorisParser.AlterRoutineLoadContext ctx) {
+ if (ctx.propertyItemList() != null) {
+ for (DorisParser.PropertyItemContext property : ctx.propertyItemList().properties) {
+ if (KafkaPropertiesUtil.isSensitiveKey(parsePropertyKey(property.key))) {
+ indexInSqlToString.put(Pair.of(property.value.start.getStartIndex(),
Review Comment:
[P1] Convert token offsets to UTF-16 before rewriting sensitive values
When the SQL contains supplementary characters, `CaseInsensitiveStream.fromString` uses ANTLR code-point indices, but `BaseViewInfo.rewriteSql` interprets these saved indices as Java UTF-16 offsets. For example, `/* 😀😀😀😀😀😀😀😀 */ ALTER ROUTINE LOAD FOR db.job FROM KAFKA ('property.sasl.password' = 'top-secret')` replaces part of the property name and leaves `-secret` in the output; with enough preceding emoji the whole password survives. This result does not throw, so both `AuditLogHelper` and FE statement logging accept the partially masked SQL. Convert both token boundaries to UTF-16 offsets against the original SQL before storing them, and cover supplementary characters before and inside sensitive values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6351221669
<details>
<summary>ClickBench: <b>Total hot run time: 24.08 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
query1 0.00 0.00 0.01
query2 0.10 0.04 0.05
query3 0.26 0.08 0.09
query4 1.61 0.10 0.09
query5 0.24 0.24 0.23
query6 1.17 0.69 0.68
query7 0.04 0.01 0.01
query8 0.05 0.03 0.03
query9 0.40 0.34 0.35
query10 0.56 0.57 0.59
query11 0.21 0.14 0.15
query12 0.18 0.15 0.15
query13 0.48 0.48 0.48
query14 1.03 1.00 1.00
query15 0.62 0.60 0.61
query16 0.35 0.35 0.34
query17 1.13 1.14 1.11
query18 0.22 0.20 0.19
query19 2.08 1.92 1.93
query20 0.02 0.01 0.02
query21 15.48 0.22 0.13
query22 4.85 0.05 0.06
query23 16.16 0.32 0.12
query24 3.02 0.55 0.33
query25 0.11 0.06 0.04
query26 1.15 0.22 0.16
query27 0.05 0.04 0.04
query28 2.74 0.66 0.37
query29 12.54 4.17 3.23
query30 0.28 0.13 0.10
query31 2.77 0.56 0.32
query32 3.22 0.59 0.50
query33 3.17 3.17 3.18
query34 15.54 3.99 3.38
query35 3.34 3.35 3.33
query36 0.55 0.44 0.43
query37 0.09 0.07 0.06
query38 0.05 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.15 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 96.23 s
Total hot run time: 24.08 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6352072835
# FE UT Coverage Report
Increment line coverage ` 85.37% (35/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69462_51c01382d4780e16cb5d712e121da86ff6ab1f66/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69462:
URL: https://github.com/apache/doris/pull/69462#discussion_r4357285059
##########
fe/fe-core/src/test/java/org/apache/doris/qe/StmtExecutorTest.java:
##########
@@ -1292,9 +1305,144 @@ public void testEmptyOriginStmtSkipsAuditMaskingReparse() throws Exception {
getStmtForLogging.setAccessible(true);
Assertions.assertEquals("", getStmtForLogging.invoke(executor, ""));
- Method getStmtForLoggingBeforeParse = StmtExecutor.class.getDeclaredMethod("getStmtForLoggingBeforeParse");
- getStmtForLoggingBeforeParse.setAccessible(true);
- Assertions.assertEquals("", getStmtForLoggingBeforeParse.invoke(executor));
+ Method getCurrentStmtForLogging = StmtExecutor.class.getDeclaredMethod("getStmtForLogging");
+ getCurrentStmtForLogging.setAccessible(true);
+ Assertions.assertEquals("", getCurrentStmtForLogging.invoke(executor));
+ }
+
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void testParsedRoutineLoadLogsMaskBothSqlModes(boolean originalNoBackslashMode) throws Exception {
+ String sql = "ALTER ROUTINE LOAD FOR db.job FROM KAFKA "
+ + "('property.sasl.\\password' = 'synthetic-value')";
+ long originalMode = connectContext.getSessionVariable().getSqlMode();
+ boolean originalPrintRequest = Config.enable_print_request_before_execution;
+ try {
+ connectContext.getSessionVariable().setSqlMode(originalNoBackslashMode
+ ? SqlModeHelper.MODE_NO_BACKSLASH_ESCAPES : SqlModeHelper.MODE_DEFAULT);
+ AlterRoutineLoadCommand command = Mockito.spy(
+ (AlterRoutineLoadCommand) new NereidsParser().parseSingle(sql));
+ // Keep real execution and its INFO/DEBUG logs; only skip the Kafka job mutation.
+ Mockito.doNothing().when(command).run(Mockito.any(), Mockito.any());
+ LogicalPlanAdapter statement = new LogicalPlanAdapter(command, new StatementContext());
+ statement.setOrigStmt(new OriginStatement(sql, 0));
+ // An earlier SET in a pre-parsed batch changes the mode before this command executes.
+ connectContext.getSessionVariable().setSqlMode(originalNoBackslashMode
+ ? SqlModeHelper.MODE_DEFAULT : SqlModeHelper.MODE_NO_BACKSLASH_ESCAPES);
+ Config.enable_print_request_before_execution = true;
+ connectContext.getState().reset();
+ try (TestLogAppender appender = TestLogAppender.attach(StmtExecutor.class)) {
+ StmtExecutor executor = new StmtExecutor(connectContext, statement);
+ executor.execute();
+
+ String expected = sql.replace("'synthetic-value'", "'******'");
+ Assertions.assertTrue(appender.contains(org.apache.logging.log4j.Level.INFO, expected));
+ Assertions.assertTrue(appender.contains(org.apache.logging.log4j.Level.DEBUG, expected));
+ Assertions.assertFalse(appender.contains(org.apache.logging.log4j.Level.INFO, "synthetic-value"));
+ Assertions.assertFalse(appender.contains(org.apache.logging.log4j.Level.DEBUG, "synthetic-value"));
+ Mockito.verify(command).run(connectContext, executor);
+ }
+ } finally {
+ Config.enable_print_request_before_execution = originalPrintRequest;
+ connectContext.getSessionVariable().setSqlMode(originalMode);
+ }
+ }
+
+ @Test
+ public void testSelectStartupLogsFailClosedOnBatchSplitMismatch() throws Exception {
+ String sql = "SELECT 1; /* outer /* inner */ x; */ ALTER ROUTINE LOAD FOR db.job FROM KAFKA "
+ + "('property.sasl.password' = 'synthetic-secret')";
+ List<StatementBase> statements = new NereidsParser().parseSQL(sql);
+ Assertions.assertEquals(2, statements.size());
+ Assertions.assertEquals(3, SqlUtils.splitMultiStmts(sql).size());
+ // ConnectProcessor retains the complete request when its splitter disagrees with Nereids.
+ StatementBase select = statements.get(0);
+ select.setOrigStmt(new OriginStatement(sql, 0));
+ boolean originalPrintRequest = Config.enable_print_request_before_execution;
+ Config.enable_print_request_before_execution = true;
+ connectContext.getState().reset();
+ try (TestLogAppender appender = TestLogAppender.attach(StmtExecutor.class)) {
+ StmtExecutor executor = new StmtExecutor(connectContext, select);
+ executor.execute();
+
+ Assertions.assertTrue(appender.contains(org.apache.logging.log4j.Level.INFO, MASKED_STMT_FALLBACK));
+ Assertions.assertTrue(appender.contains(org.apache.logging.log4j.Level.DEBUG, MASKED_STMT_FALLBACK));
+ Assertions.assertFalse(appender.contains(org.apache.logging.log4j.Level.INFO, "synthetic-secret"));
+ Assertions.assertFalse(appender.contains(org.apache.logging.log4j.Level.DEBUG, "synthetic-secret"));
+ } finally {
+ Config.enable_print_request_before_execution = originalPrintRequest;
+ }
+ }
+
+ @Test
+ public void testForwardedRoutineLoadAuditMasksMasterKeyInterpretation() throws Exception {
+ String alter = "ALTER ROUTINE LOAD FOR testDb.job FROM KAFKA "
+ + "('property.sasl.\\password' = 'synthetic-secret', 'property.client.id' = 'client-value')";
+ String batch = "SET sql_mode=''; " + alter;
+ ConnectContext follower = Mockito.spy(new ConnectContext());
+ follower.setCurrentUserIdentity(UserIdentity.ROOT);
+ follower.setEnv(Env.getCurrentEnv());
+ follower.setRemoteIP("127.0.0.1");
+ follower.setStartTime();
+ follower.getSessionVariable().setSqlMode(SqlModeHelper.MODE_NO_BACKSLASH_ESCAPES);
+ follower.setThreadLocalInfo();
+ Mockito.doReturn(null).when(follower).getCurrentCatalog();
+ try {
+ // Parse the whole batch before running its SET, as ConnectProcessor does.
+ List<StatementBase> statements = new NereidsParser().parseSQL(batch);
+ LogicalPlanAdapter followerStatement = (LogicalPlanAdapter) statements.get(1);
+ followerStatement.setOrigStmt(new OriginStatement(alter, 0));
+ AlterRoutineLoadCommand followerCommand = (AlterRoutineLoadCommand) followerStatement.getLogicalPlan();
+ Assertions.assertEquals("synthetic-secret",
+ followerCommand.getDataSourceMapProperties().get("property.sasl.\\password"));
+ LogicalPlanAdapter setStatement = (LogicalPlanAdapter) statements.get(0);
+ follower.setStatementContext(setStatement.getStatementContext());
+ ((SetOptionsCommand) setStatement.getLogicalPlan()).run(follower, null);
+ Assertions.assertEquals(SqlModeHelper.MODE_DEFAULT, follower.getSessionVariable().getSqlMode());
Review Comment:
[P2] Expect zero after clearing sql_mode in the forwarding test
The preceding `SET sql_mode=''` is executed by the real `SetOptionsCommand`, whose `VariableMgr` converter calls `SqlModeHelper.encode("")` and stores `0L`. `MODE_DEFAULT` is `1L`, and neither the field assignment nor callbacks normalize zero to that value, so this assertion fails before the request serialization and audit checks run. Expect `0L` (or assert that `NO_BACKSLASH_ESCAPES` is disabled) so the new regression reaches the behavior it is meant to cover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6365721338
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6382974813
<details>
<summary>TPC-H: <b>Total hot run time: 28163 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3eb671efdd667136aea10b75b7a229fe8c820a1d, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17623 5146 5048 5048
q2 2094 310 251 251
q3 10261 1362 793 793
q4 4673 456 336 336
q5 7433 798 533 533
q6 179 185 152 152
q7 734 783 594 594
q8 9251 1293 1026 1026
q9 5835 4187 4140 4140
q10 6832 1275 1003 1003
q11 425 247 223 223
q12 636 391 273 273
q13 18126 3546 2999 2999
q14 274 278 252 252
q15 q16 714 712 678 678
q17 1196 744 564 564
q18 6930 6143 6108 6108
q19 1103 948 585 585
q20 373 323 214 214
q21 5302 2147 2279 2147
q22 349 291 244 244
Total cold run time: 100343 ms
Total hot run time: 28163 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5619 5596 5597 5596
q2 663 545 537 537
q3 5204 5370 4884 4884
q4 1976 2139 1532 1532
q5 5290 5167 5512 5167
q6 262 203 150 150
q7 2244 2074 1921 1921
q8 3004 2537 2653 2537
q9 8079 7943 7967 7943
q10 3716 3691 3198 3198
q11 598 586 416 416
q12 696 673 530 530
q13 3461 3901 3223 3223
q14 322 300 284 284
q15 q16 727 733 644 644
q17 8245 7370 6856 6856
q18 13019 12128 12854 12128
q19 887 775 777 775
q20 2133 2144 1931 1931
q21 5592 4579 4648 4579
q22 475 438 404 404
Total cold run time: 72212 ms
Total hot run time: 65235 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6383065866
<details>
<summary>TPC-DS: <b>Total hot run time: 150460 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3eb671efdd667136aea10b75b7a229fe8c820a1d, data reload: false
query5 4307 547 451 451
query6 415 208 184 184
query7 4804 440 220 220
query8 330 166 145 145
query9 8770 3042 3057 3042
query10 469 312 256 256
query11 6020 3844 3561 3561
query12 145 89 88 88
query13 1234 448 305 305
query14 6100 3840 3601 3601
query14_1 3400 3355 3401 3355
query15 213 203 185 185
query16 965 442 400 400
query17 890 637 513 513
query18 2413 406 306 306
query19 195 155 127 127
query20 81 80 79 79
query21 209 123 107 107
query22 12880 12985 12883 12883
query23 13270 12790 12583 12583
query23_1 12459 12687 12487 12487
query24 7387 803 448 448
query24_1 465 450 456 450
query25 520 392 341 341
query26 1250 255 135 135
query27 2790 441 273 273
query28 5405 1923 1906 1906
query29 1546 581 418 418
query30 299 223 180 180
query31 1009 916 810 810
query32 141 90 91 90
query33 523 289 232 232
query34 963 854 509 509
query35 723 767 634 634
query36 820 792 712 712
query37 120 99 77 77
query38 1759 1694 1656 1656
query39 788 777 745 745
query39_1 719 721 724 721
query40 214 112 90 90
query41 65 63 61 61
query42 82 86 79 79
query43 338 363 308 308
query44 1279 672 660 660
query45 174 170 165 165
query46 823 949 552 552
query47 2880 2807 2822 2807
query48 286 300 206 206
query49 569 392 290 290
query50 665 268 212 212
query51 10850 10204 10294 10204
query52 77 79 69 69
query53 182 204 155 155
query54 238 183 169 169
query55 70 63 63 63
query56 219 210 223 210
query57 1656 1647 1545 1545
query58 280 269 242 242
query59 2168 2236 2017 2017
query60 293 259 225 225
query61 148 148 134 134
query62 392 329 283 283
query63 187 153 160 153
query64 2758 1063 861 861
query65 3312 3270 3279 3270
query66 1796 432 314 314
query67 20061 20039 20057 20039
query68 2964 993 603 603
query69 380 283 256 256
query70 832 810 773 773
query71 301 240 217 217
query72 2861 2648 2447 2447
query73 511 524 291 291
query74 4617 4542 4343 4343
query75 2313 2248 1903 1903
query76 2305 1002 578 578
query77 341 370 259 259
query78 9868 9769 9154 9154
query79 974 853 509 509
query80 1138 422 343 343
query81 591 315 274 274
query82 571 130 106 106
query83 290 193 179 179
query84 319 120 95 95
query85 845 416 361 361
query86 375 232 226 226
query87 1999 1934 1789 1789
query88 3585 2630 2631 2630
query89 326 280 245 245
query90 1846 166 163 163
query91 163 151 125 125
query92 95 85 82 82
query93 993 980 568 568
query94 568 317 321 317
query95 585 322 301 301
query96 656 476 232 232
query97 2476 2439 2348 2348
query98 157 152 147 147
query99 774 773 657 657
Total cold run time: 235140 ms
Total hot run time: 150460 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6347368808
<details>
<summary>TPC-DS: <b>Total hot run time: 150361 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51c01382d4780e16cb5d712e121da86ff6ab1f66, data reload: false
query5 4301 588 456 456
query6 421 205 195 195
query7 4797 437 231 231
query8 318 168 148 148
query9 8762 3135 3119 3119
query10 479 303 261 261
query11 6082 3873 3591 3591
query12 146 89 86 86
query13 1254 425 311 311
query14 6100 3950 3591 3591
query14_1 3436 3431 3428 3428
query15 211 198 183 183
query16 955 400 394 394
query17 847 649 540 540
query18 2409 423 312 312
query19 195 164 133 133
query20 86 85 81 81
query21 221 127 107 107
query22 12895 12961 12795 12795
query23 13448 12879 12438 12438
query23_1 12553 12528 12655 12528
query24 7372 829 472 472
query24_1 473 474 483 474
query25 538 423 361 361
query26 1309 258 139 139
query27 2773 434 279 279
query28 4622 1914 1921 1914
query29 1576 579 455 455
query30 308 220 183 183
query31 997 931 797 797
query32 140 86 83 83
query33 519 294 234 234
query34 894 843 494 494
query35 729 754 631 631
query36 818 829 712 712
query37 122 97 84 84
query38 1759 1728 1660 1660
query39 778 809 738 738
query39_1 722 721 726 721
query40 224 116 94 94
query41 66 61 60 60
query42 84 82 81 81
query43 338 365 308 308
query44 1281 688 710 688
query45 179 184 158 158
query46 831 962 557 557
query47 3014 2997 2783 2783
query48 292 304 218 218
query49 568 395 303 303
query50 678 290 210 210
query51 10476 10487 10358 10358
query52 76 78 68 68
query53 186 202 147 147
query54 234 199 176 176
query55 68 64 67 64
query56 220 232 214 214
query57 1639 1622 1572 1572
query58 288 254 247 247
query59 2061 2096 1844 1844
query60 294 242 223 223
query61 144 144 144 144
query62 390 326 294 294
query63 180 155 159 155
query64 2712 898 752 752
query65 3301 3244 3244 3244
query66 1776 416 293 293
query67 20297 20087 19810 19810
query68 2939 1001 603 603
query69 383 306 242 242
query70 853 771 765 765
query71 300 238 215 215
query72 2734 2526 2381 2381
query73 513 525 301 301
query74 4588 4511 4334 4334
query75 2340 2309 1916 1916
query76 2333 1028 614 614
query77 344 367 262 262
query78 9700 9670 9189 9189
query79 979 870 519 519
query80 560 445 364 364
query81 536 332 286 286
query82 579 133 114 114
query83 302 196 188 188
query84 323 127 113 113
query85 936 424 359 359
query86 321 238 227 227
query87 2002 1965 1779 1779
query88 3624 2689 2673 2673
query89 327 286 243 243
query90 1895 164 165 164
query91 161 147 131 131
query92 93 88 92 88
query93 981 998 571 571
query94 431 305 271 271
query95 569 387 300 300
query96 670 483 229 229
query97 2433 2429 2312 2312
query98 163 150 158 150
query99 768 763 651 651
Total cold run time: 233548 ms
Total hot run time: 150361 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) Avoid logging Kafka Routine Load property values [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6337688421
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6359801578
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6357452127
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6364704907
<details>
<summary>TPC-H: <b>Total hot run time: 28566 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4904130dad6ac79752a830c63d9ddab1c78e0480, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17616 5132 5043 5043
q2 2114 319 270 270
q3 10204 1353 829 829
q4 4688 464 337 337
q5 7423 790 537 537
q6 182 189 155 155
q7 754 812 588 588
q8 9243 1332 1034 1034
q9 5402 4180 4145 4145
q10 6813 1277 1039 1039
q11 425 244 233 233
q12 643 394 281 281
q13 18143 3561 2996 2996
q14 287 274 243 243
q15 q16 752 715 663 663
q17 1200 740 573 573
q18 6936 6182 6141 6141
q19 1095 955 594 594
q20 400 330 225 225
q21 5208 2449 2363 2363
q22 388 329 277 277
Total cold run time: 99916 ms
Total hot run time: 28566 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6012 5895 6063 5895
q2 734 613 576 576
q3 5559 5668 5314 5314
q4 2028 2151 1797 1797
q5 5656 5587 5957 5587
q6 247 188 138 138
q7 2175 1976 1788 1788
q8 2949 2625 2641 2625
q9 7947 7935 7782 7782
q10 3631 3572 3172 3172
q11 580 413 399 399
q12 660 663 544 544
q13 3276 3615 3033 3033
q14 278 295 259 259
q15 q16 651 670 612 612
q17 7740 6977 6859 6859
q18 12965 12149 12825 12149
q19 914 786 781 781
q20 2157 2132 1917 1917
q21 5552 4578 4666 4578
q22 495 443 410 410
Total cold run time: 72206 ms
Total hot run time: 66215 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6382685475
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6364883887
<details>
<summary>TPC-DS: <b>Total hot run time: 150377 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4904130dad6ac79752a830c63d9ddab1c78e0480, data reload: false
query5 4323 584 431 431
query6 450 208 203 203
query7 4790 446 221 221
query8 318 163 154 154
query9 8765 3053 3052 3052
query10 477 313 258 258
query11 5962 3836 3593 3593
query12 146 90 86 86
query13 1249 428 313 313
query14 6153 3873 3609 3609
query14_1 3398 3424 3361 3361
query15 207 196 182 182
query16 950 430 403 403
query17 873 651 535 535
query18 2430 427 316 316
query19 220 171 145 145
query20 87 92 84 84
query21 208 137 106 106
query22 12956 12988 12759 12759
query23 13413 12815 12514 12514
query23_1 12616 12547 12563 12547
query24 7334 811 458 458
query24_1 461 474 480 474
query25 529 424 354 354
query26 1308 256 143 143
query27 2743 436 274 274
query28 4607 1909 1908 1908
query29 1595 615 446 446
query30 300 227 192 192
query31 1012 931 815 815
query32 153 97 96 96
query33 531 308 246 246
query34 869 838 503 503
query35 733 747 653 653
query36 828 805 745 745
query37 130 101 87 87
query38 1768 1756 1674 1674
query39 789 787 743 743
query39_1 734 725 757 725
query40 224 115 96 96
query41 73 69 68 68
query42 94 85 87 85
query43 342 361 306 306
query44 1315 687 684 684
query45 179 176 178 176
query46 819 968 566 566
query47 2959 2889 2593 2593
query48 295 299 217 217
query49 578 396 286 286
query50 665 286 196 196
query51 10567 10425 10649 10425
query52 79 78 71 71
query53 185 203 151 151
query54 249 196 190 190
query55 73 63 58 58
query56 245 205 219 205
query57 1622 1607 1568 1568
query58 282 261 254 254
query59 2149 2147 1972 1972
query60 286 245 224 224
query61 148 146 144 144
query62 390 334 281 281
query63 183 153 156 153
query64 2730 971 780 780
query65 3319 3266 3257 3257
query66 1789 417 302 302
query67 20170 20044 19991 19991
query68 3099 982 601 601
query69 372 278 258 258
query70 805 748 783 748
query71 291 226 216 216
query72 2818 2714 2281 2281
query73 517 522 291 291
query74 4657 4502 4342 4342
query75 2365 2284 1904 1904
query76 2326 990 582 582
query77 335 364 266 266
query78 9869 9666 9255 9255
query79 973 846 510 510
query80 1240 460 333 333
query81 599 313 270 270
query82 848 127 102 102
query83 304 194 185 185
query84 313 125 106 106
query85 869 430 376 376
query86 402 239 233 233
query87 1997 1951 1780 1780
query88 3627 2631 2615 2615
query89 336 277 238 238
query90 1909 169 162 162
query91 165 151 125 125
query92 97 93 89 89
query93 1002 980 575 575
query94 651 288 269 269
query95 583 385 303 303
query96 669 473 231 231
query97 2442 2426 2303 2303
query98 161 150 145 145
query99 770 763 651 651
Total cold run time: 235619 ms
Total hot run time: 150377 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Mask Kafka Routine Load credentials in logs and audit SQL [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69462:
URL: https://github.com/apache/doris/pull/69462#issuecomment-6383111629
<details>
<summary>ClickBench: <b>Total hot run time: 24.2 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3eb671efdd667136aea10b75b7a229fe8c820a1d, data reload: false
query1 0.01 0.01 0.01
query2 0.09 0.05 0.05
query3 0.26 0.08 0.07
query4 1.62 0.10 0.10
query5 0.23 0.24 0.24
query6 1.17 0.68 0.68
query7 0.03 0.01 0.00
query8 0.05 0.04 0.04
query9 0.40 0.36 0.33
query10 0.57 0.56 0.58
query11 0.21 0.14 0.13
query12 0.17 0.16 0.14
query13 0.48 0.50 0.49
query14 1.04 1.02 0.99
query15 0.61 0.60 0.61
query16 0.35 0.36 0.35
query17 1.13 1.20 1.16
query18 0.20 0.20 0.19
query19 2.09 1.99 2.01
query20 0.02 0.02 0.01
query21 15.48 0.19 0.13
query22 4.94 0.06 0.05
query23 16.29 0.33 0.12
query24 2.98 0.49 0.32
query25 0.12 0.07 0.04
query26 1.25 0.22 0.16
query27 0.05 0.05 0.05
query28 3.82 0.78 0.35
query29 12.49 4.13 3.22
query30 0.28 0.14 0.10
query31 2.77 0.56 0.32
query32 3.22 0.60 0.50
query33 3.23 3.23 3.22
query34 15.31 4.00 3.38
query35 3.34 3.37 3.35
query36 0.55 0.43 0.42
query37 0.10 0.06 0.06
query38 0.06 0.04 0.03
query39 0.04 0.03 0.03
query40 0.16 0.14 0.15
query41 0.08 0.03 0.02
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.37 s
Total hot run time: 24.2 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org