You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morrySnow (via GitHub)" <gi...@apache.org> on 2024/06/24 03:47:58 UTC
[PR] [opt](persist) do not persist variable with default value [doris]
morrySnow opened a new pull request, #36728:
URL: https://github.com/apache/doris/pull/36728
do not persist variable with default value anymore. During version upgrade, it is easy to change the default value of session variables and make it effective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist boolean variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2326895548
<details>
<summary>TPC-H: <b>Total hot run time: 38204 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 207ce0101a86798504f032c546704e6c802aa120, data reload: false
------ Round 1 ----------------------------------
q1 17479 4533 4402 4402
q2 1871 210 204 204
q3 10155 1163 1088 1088
q4 9997 794 718 718
q5 7682 2828 2747 2747
q6 227 136 138 136
q7 962 620 595 595
q8 9247 2089 2064 2064
q9 7286 6541 6577 6541
q10 7010 2164 2293 2164
q11 456 240 250 240
q12 400 218 221 218
q13 17779 3097 3067 3067
q14 282 234 235 234
q15 524 496 481 481
q16 576 501 498 498
q17 982 613 769 613
q18 7373 7013 6897 6897
q19 1393 1042 1098 1042
q20 691 342 329 329
q21 4024 3124 2926 2926
q22 1122 1005 1000 1000
Total cold run time: 107518 ms
Total hot run time: 38204 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4413 4379 4314 4314
q2 382 279 279 279
q3 2857 2663 2665 2663
q4 1915 1631 1621 1621
q5 5502 5648 5746 5648
q6 227 134 133 133
q7 2232 1874 1816 1816
q8 3311 3541 3497 3497
q9 8838 8888 8769 8769
q10 3597 3381 3374 3374
q11 608 520 517 517
q12 851 659 678 659
q13 17245 3300 3326 3300
q14 337 306 318 306
q15 540 473 479 473
q16 637 592 584 584
q17 1859 1565 1572 1565
q18 8115 7889 7911 7889
q19 1718 1558 1542 1542
q20 2160 1827 1838 1827
q21 5602 5347 5296 5296
q22 1159 1080 1050 1050
Total cold run time: 74105 ms
Total hot run time: 57122 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist boolean variable with default value [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2326856366
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2185530138
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist boolean variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2326939649
<details>
<summary>ClickBench: <b>Total hot run time: 31.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 207ce0101a86798504f032c546704e6c802aa120, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.03 0.04
query3 0.23 0.04 0.05
query4 1.69 0.08 0.07
query5 0.51 0.47 0.50
query6 1.13 0.72 0.74
query7 0.02 0.01 0.02
query8 0.04 0.05 0.05
query9 0.55 0.48 0.49
query10 0.53 0.54 0.54
query11 0.16 0.12 0.12
query12 0.15 0.13 0.13
query13 0.61 0.59 0.59
query14 2.06 2.10 2.04
query15 0.87 0.82 0.83
query16 0.35 0.37 0.35
query17 1.00 1.01 1.02
query18 0.21 0.20 0.20
query19 1.82 1.76 1.76
query20 0.01 0.01 0.01
query21 15.39 0.67 0.65
query22 4.10 7.80 1.75
query23 18.25 1.32 1.28
query24 2.09 0.22 0.22
query25 0.14 0.08 0.07
query26 0.26 0.19 0.18
query27 0.08 0.07 0.08
query28 13.21 1.01 1.02
query29 12.60 3.33 3.32
query30 0.24 0.05 0.06
query31 2.87 0.40 0.39
query32 3.27 0.49 0.48
query33 2.95 2.96 3.04
query34 17.15 4.39 4.44
query35 4.48 4.50 4.49
query36 0.67 0.50 0.48
query37 0.19 0.15 0.16
query38 0.15 0.14 0.14
query39 0.05 0.04 0.04
query40 0.15 0.13 0.12
query41 0.10 0.05 0.05
query42 0.05 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.53 s
Total hot run time: 31.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist boolean variable with default value [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow closed pull request #36728: [opt](persist) do not persist boolean variable with default value
URL: https://github.com/apache/doris/pull/36728
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2185617611
<details>
<summary>TPC-DS: <b>Total hot run time: 174371 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b16e1f50404d15c93e4ec08a33ff4b0c25e6d49, data reload: false
query1 909 384 370 370
query2 6448 2441 2417 2417
query3 6631 207 223 207
query4 19337 17208 17324 17208
query5 3677 459 463 459
query6 235 176 160 160
query7 4583 294 287 287
query8 320 280 288 280
query9 8635 2376 2360 2360
query10 558 325 271 271
query11 10559 10007 10041 10007
query12 111 92 85 85
query13 1654 381 388 381
query14 9765 7821 7236 7236
query15 231 192 187 187
query16 7821 282 263 263
query17 1414 534 518 518
query18 1957 275 273 273
query19 201 148 154 148
query20 84 81 80 80
query21 208 127 124 124
query22 4140 4034 3918 3918
query23 33839 33868 33752 33752
query24 11350 2834 2905 2834
query25 635 419 380 380
query26 710 160 161 160
query27 2349 336 324 324
query28 6070 2115 2117 2115
query29 897 627 635 627
query30 232 162 152 152
query31 960 760 759 759
query32 93 53 54 53
query33 752 282 296 282
query34 1019 495 500 495
query35 756 658 642 642
query36 1131 981 1010 981
query37 152 77 75 75
query38 2957 2830 2893 2830
query39 938 851 816 816
query40 215 136 143 136
query41 60 58 54 54
query42 113 103 99 99
query43 575 561 528 528
query44 1232 758 742 742
query45 203 169 165 165
query46 1083 742 750 742
query47 1837 1778 1754 1754
query48 383 311 316 311
query49 832 420 417 417
query50 769 397 397 397
query51 6822 6782 6785 6782
query52 108 95 91 91
query53 367 307 287 287
query54 880 448 445 445
query55 76 75 78 75
query56 288 266 267 266
query57 1143 1087 1043 1043
query58 260 238 237 237
query59 3442 3182 3166 3166
query60 290 284 281 281
query61 95 92 119 92
query62 626 471 460 460
query63 319 302 301 301
query64 8529 2241 1749 1749
query65 3199 3099 3094 3094
query66 750 321 333 321
query67 15486 15039 14900 14900
query68 4716 555 555 555
query69 596 461 380 380
query70 1195 1132 1083 1083
query71 448 282 283 282
query72 7378 5453 5618 5453
query73 770 333 330 330
query74 5862 5640 5534 5534
query75 3497 2670 2722 2670
query76 2797 934 923 923
query77 612 318 311 311
query78 10243 9816 9675 9675
query79 2701 529 535 529
query80 1960 537 478 478
query81 550 216 225 216
query82 1098 111 102 102
query83 339 172 173 172
query84 268 89 89 89
query85 1349 278 276 276
query86 471 283 309 283
query87 3331 3122 3099 3099
query88 4339 2468 2442 2442
query89 484 375 384 375
query90 1707 195 192 192
query91 128 107 102 102
query92 62 51 52 51
query93 2582 513 501 501
query94 1115 189 187 187
query95 405 316 322 316
query96 615 273 272 272
query97 3223 3041 3035 3035
query98 222 203 194 194
query99 1253 813 836 813
Total cold run time: 270616 ms
Total hot run time: 174371 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2185530180
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2185623203
<details>
<summary>ClickBench: <b>Total hot run time: 30.73 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b16e1f50404d15c93e4ec08a33ff4b0c25e6d49, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.04 0.03
query3 0.23 0.05 0.05
query4 1.68 0.06 0.06
query5 0.51 0.49 0.52
query6 1.14 0.72 0.72
query7 0.02 0.01 0.02
query8 0.06 0.03 0.04
query9 0.55 0.50 0.49
query10 0.56 0.55 0.54
query11 0.15 0.12 0.11
query12 0.15 0.12 0.12
query13 0.60 0.59 0.58
query14 0.76 0.78 0.78
query15 0.83 0.82 0.82
query16 0.36 0.36 0.37
query17 1.02 1.01 1.03
query18 0.22 0.22 0.24
query19 1.81 1.70 1.80
query20 0.02 0.01 0.01
query21 15.40 0.66 0.67
query22 4.17 7.14 2.18
query23 18.31 1.42 1.27
query24 2.14 0.24 0.22
query25 0.15 0.09 0.09
query26 0.26 0.18 0.17
query27 0.08 0.08 0.08
query28 13.23 1.03 1.01
query29 12.62 3.27 3.22
query30 0.26 0.06 0.06
query31 2.87 0.39 0.39
query32 3.28 0.47 0.47
query33 2.88 2.86 2.95
query34 17.11 4.38 4.37
query35 4.47 4.46 4.56
query36 0.66 0.47 0.47
query37 0.17 0.16 0.16
query38 0.16 0.15 0.14
query39 0.04 0.03 0.03
query40 0.16 0.14 0.15
query41 0.10 0.05 0.04
query42 0.06 0.04 0.05
query43 0.04 0.04 0.05
Total cold run time: 109.42 s
Total hot run time: 30.73 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist boolean variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2326924251
<details>
<summary>TPC-DS: <b>Total hot run time: 192721 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 207ce0101a86798504f032c546704e6c802aa120, data reload: false
query1 1267 899 858 858
query2 6056 2067 2010 2010
query3 10683 4244 3995 3995
query4 59587 26382 23148 23148
query5 5397 505 492 492
query6 413 170 168 168
query7 5781 294 290 290
query8 292 211 221 211
query9 8961 2491 2471 2471
query10 492 292 266 266
query11 18107 15061 15323 15061
query12 152 110 104 104
query13 1577 402 398 398
query14 10637 7534 7417 7417
query15 224 181 178 178
query16 7429 502 497 497
query17 1139 621 557 557
query18 1878 300 291 291
query19 289 147 144 144
query20 128 109 108 108
query21 217 106 102 102
query22 4886 4653 4624 4624
query23 34191 33370 33342 33342
query24 5904 2899 2929 2899
query25 513 393 375 375
query26 672 151 153 151
query27 1757 265 274 265
query28 3934 2027 1996 1996
query29 657 395 400 395
query30 226 152 154 152
query31 922 754 774 754
query32 74 53 53 53
query33 469 290 277 277
query34 866 468 463 463
query35 845 711 709 709
query36 1056 944 937 937
query37 148 86 87 86
query38 3984 3871 3941 3871
query39 1443 1425 1428 1425
query40 199 113 111 111
query41 46 46 45 45
query42 118 92 95 92
query43 532 480 477 477
query44 1059 723 722 722
query45 194 166 161 161
query46 1085 748 776 748
query47 1901 1795 1820 1795
query48 374 298 289 289
query49 756 413 422 413
query50 809 414 405 405
query51 7077 6808 6808 6808
query52 99 90 87 87
query53 248 176 171 171
query54 559 440 446 440
query55 75 74 73 73
query56 285 258 259 258
query57 1176 1091 1093 1091
query58 216 215 238 215
query59 3142 2874 2888 2874
query60 291 270 274 270
query61 107 99 136 99
query62 745 643 664 643
query63 208 176 186 176
query64 2484 657 669 657
query65 3206 3142 3132 3132
query66 680 334 337 334
query67 15446 15271 15296 15271
query68 2998 561 569 561
query69 406 278 279 278
query70 1161 1180 1000 1000
query71 360 325 271 271
query72 6183 3992 3993 3992
query73 751 324 323 323
query74 9034 8687 8766 8687
query75 3361 2678 2732 2678
query76 1364 951 992 951
query77 557 310 319 310
query78 10722 9240 9012 9012
query79 1050 530 514 514
query80 706 507 529 507
query81 515 236 237 236
query82 242 151 147 147
query83 169 148 149 148
query84 261 78 76 76
query85 680 288 300 288
query86 300 298 294 294
query87 4499 4229 4303 4229
query88 3112 2331 2339 2331
query89 375 286 285 285
query90 1960 188 187 187
query91 124 102 103 102
query92 136 51 51 51
query93 1031 517 518 517
query94 747 282 300 282
query95 341 254 259 254
query96 590 265 266 265
query97 3218 3140 3071 3071
query98 212 210 199 199
query99 1506 1278 1304 1278
Total cold run time: 305041 ms
Total hot run time: 192721 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #36728:
URL: https://github.com/apache/doris/pull/36728#discussion_r1694453088
##########
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##########
@@ -3432,6 +3434,11 @@ public JSONObject toJson() throws IOException {
if (attr == null) {
continue;
}
+ String defaultValue = VariableMgr.getDefaultValue(attr.name());
+ String currentValue = VariableMgr.getValue(this, new VariableExpr(attr.name(), SetType.SESSION));
+ if (defaultValue != null && defaultValue.equals(currentValue)) {
Review Comment:
If a user explicitly sets a value and it equals the default value, will it not be persisted? Then after an upgrade, if the default value changes, will the user's set value be 'changed' to the new one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](persist) do not persist variable with default value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36728:
URL: https://github.com/apache/doris/pull/36728#issuecomment-2185606750
<details>
<summary>TPC-H: <b>Total hot run time: 39659 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b16e1f50404d15c93e4ec08a33ff4b0c25e6d49, data reload: false
------ Round 1 ----------------------------------
q1 17607 4556 4247 4247
q2 2017 199 189 189
q3 10461 1110 1077 1077
q4 10189 787 761 761
q5 7454 2670 2640 2640
q6 220 133 134 133
q7 938 607 596 596
q8 9215 2052 2050 2050
q9 9120 6469 6428 6428
q10 8944 3713 3724 3713
q11 454 240 235 235
q12 487 231 228 228
q13 17773 2979 2989 2979
q14 260 227 214 214
q15 524 498 500 498
q16 515 374 370 370
q17 966 672 693 672
q18 8032 7529 7326 7326
q19 7326 1415 1521 1415
q20 647 327 328 327
q21 4848 3228 3863 3228
q22 402 335 333 333
Total cold run time: 118399 ms
Total hot run time: 39659 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4415 4229 4210 4210
q2 383 267 274 267
q3 2943 2901 2939 2901
q4 1983 1678 1749 1678
q5 5562 5518 5438 5438
q6 223 131 130 130
q7 2260 1865 1847 1847
q8 3281 3446 3434 3434
q9 8716 8732 8780 8732
q10 4153 3792 3754 3754
q11 590 496 525 496
q12 829 619 640 619
q13 16009 3190 3213 3190
q14 307 274 285 274
q15 524 500 465 465
q16 483 435 460 435
q17 1819 1501 1546 1501
q18 8210 7947 7952 7947
q19 1798 1551 1733 1551
q20 3197 1896 1888 1888
q21 5130 5014 4653 4653
q22 613 567 534 534
Total cold run time: 73428 ms
Total hot run time: 55944 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org