You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/09/13 10:51:51 UTC
[PR] [improve](restore) Disable alter ops during atomic restore [doris]
w41ter opened a new pull request, #40817:
URL: https://github.com/apache/doris/pull/40817
The atomic restore is introduced in #40353. It replaces tables instead of updating in place, but the origin table might be altered during the restoring, which might cause the restore job to fail. This PR adds a property in_atomic_restore for those tables and forbids any update to those tables involved in restoring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #40817:
URL: https://github.com/apache/doris/pull/40817
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2348725088
<details>
<summary>TPC-DS: <b>Total hot run time: 194557 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc406246e24a3ba8aec0ede744b8c4823cc60473, data reload: false
query1 932 384 375 375
query2 6462 1743 1746 1743
query3 6655 209 219 209
query4 26245 24282 23869 23869
query5 4998 541 510 510
query6 263 177 167 167
query7 4587 307 305 305
query8 284 225 225 225
query9 8508 2586 2601 2586
query10 458 277 279 277
query11 16116 15714 15589 15589
query12 166 102 101 101
query13 1688 405 381 381
query14 11412 6483 6730 6483
query15 217 174 172 172
query16 7616 476 463 463
query17 1540 586 569 569
query18 1986 296 295 295
query19 204 153 155 153
query20 120 111 114 111
query21 209 108 106 106
query22 4402 4348 4330 4330
query23 34499 33665 33562 33562
query24 9752 3116 3086 3086
query25 681 411 417 411
query26 1376 164 161 161
query27 2907 288 294 288
query28 6928 2117 2090 2090
query29 974 432 429 429
query30 301 162 152 152
query31 984 794 820 794
query32 104 58 61 58
query33 731 317 316 316
query34 903 496 485 485
query35 856 733 752 733
query36 1091 914 938 914
query37 174 86 87 86
query38 4024 3974 3870 3870
query39 1447 1406 1424 1406
query40 286 120 119 119
query41 51 48 50 48
query42 122 98 102 98
query43 502 445 429 429
query44 1242 798 780 780
query45 202 184 173 173
query46 1115 844 813 813
query47 1942 1803 1873 1803
query48 366 290 296 290
query49 1135 452 459 452
query50 915 444 459 444
query51 7086 7152 6948 6948
query52 99 90 90 90
query53 264 199 189 189
query54 759 485 477 477
query55 80 77 79 77
query56 298 286 271 271
query57 1235 1085 1049 1049
query58 261 272 259 259
query59 2765 2862 2455 2455
query60 325 293 292 292
query61 130 224 105 105
query62 918 695 675 675
query63 224 194 189 189
query64 5288 677 679 677
query65 3250 3210 3217 3210
query66 1417 298 305 298
query67 16093 15643 15406 15406
query68 3897 606 570 570
query69 415 289 272 272
query70 1195 1140 1036 1036
query71 348 283 287 283
query72 6381 4174 4057 4057
query73 764 332 333 332
query74 9571 9329 9015 9015
query75 3395 2750 2768 2750
query76 2582 1262 1316 1262
query77 453 330 332 330
query78 9812 9541 9284 9284
query79 1105 890 884 884
query80 976 822 819 819
query81 509 261 269 261
query82 1373 262 261 261
query83 208 196 195 195
query84 235 109 106 106
query85 723 431 388 388
query86 326 307 307 307
query87 4376 4313 4317 4313
query88 4133 4057 4058 4057
query89 370 368 369 368
query90 1740 336 313 313
query91 123 123 123 123
query92 78 74 75 74
query93 1048 1058 1046 1046
query94 853 380 358 358
query95 500 415 420 415
query96 469 468 468 468
query97 3187 3114 3158 3114
query98 226 227 226 226
query99 1594 1319 1294 1294
Total cold run time: 284135 ms
Total hot run time: 194557 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2348744492
<details>
<summary>ClickBench: <b>Total hot run time: 31.13 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc406246e24a3ba8aec0ede744b8c4823cc60473, data reload: false
query1 0.05 0.05 0.04
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.69 0.08 0.08
query5 0.52 0.49 0.49
query6 1.14 0.73 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.56 0.54 0.51
query10 0.57 0.59 0.56
query11 0.16 0.12 0.11
query12 0.16 0.13 0.13
query13 0.63 0.61 0.61
query14 1.46 1.52 1.49
query15 0.91 0.87 0.88
query16 0.37 0.36 0.36
query17 1.02 1.04 1.04
query18 0.22 0.21 0.21
query19 1.95 1.78 1.86
query20 0.01 0.00 0.01
query21 15.39 0.70 0.68
query22 4.87 8.19 1.22
query23 17.96 1.34 1.33
query24 2.23 0.23 0.22
query25 0.18 0.08 0.09
query26 0.29 0.18 0.19
query27 0.09 0.07 0.08
query28 13.19 1.14 1.11
query29 12.52 3.34 3.34
query30 0.25 0.06 0.05
query31 2.86 0.43 0.42
query32 3.23 0.50 0.50
query33 3.05 3.02 3.06
query34 15.45 4.32 4.33
query35 4.35 4.34 4.34
query36 0.68 0.49 0.50
query37 0.19 0.17 0.16
query38 0.17 0.15 0.15
query39 0.05 0.04 0.04
query40 0.16 0.13 0.14
query41 0.10 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 109.14 s
Total hot run time: 31.13 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2348648527
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2349167864
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2348647970
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2348700572
<details>
<summary>TPC-H: <b>Total hot run time: 43041 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc406246e24a3ba8aec0ede744b8c4823cc60473, data reload: false
------ Round 1 ----------------------------------
q1 18092 7386 7277 7277
q2 2685 195 187 187
q3 11121 1406 1385 1385
q4 12046 993 1077 993
q5 7785 3217 3137 3137
q6 237 151 149 149
q7 1038 635 612 612
q8 9467 2020 2037 2020
q9 6733 6312 6343 6312
q10 7006 2504 2555 2504
q11 438 255 256 255
q12 409 228 222 222
q13 17766 3020 3014 3014
q14 286 253 255 253
q15 597 533 527 527
q16 514 426 417 417
q17 999 955 945 945
q18 7368 6779 6867 6779
q19 1386 1228 1242 1228
q20 606 352 332 332
q21 3890 3526 3509 3509
q22 1047 984 998 984
Total cold run time: 111516 ms
Total hot run time: 43041 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7184 7149 7224 7149
q2 341 229 231 229
q3 2889 2868 2843 2843
q4 1948 1944 1944 1944
q5 5448 5405 5422 5405
q6 234 145 143 143
q7 2025 1701 1676 1676
q8 3228 3293 3301 3293
q9 8427 8381 8386 8381
q10 3381 3457 3444 3444
q11 575 468 464 464
q12 775 568 620 568
q13 5405 3017 3022 3017
q14 299 263 281 263
q15 557 522 525 522
q16 489 439 445 439
q17 1744 1696 1695 1695
q18 8094 7484 7725 7484
q19 1708 1699 1689 1689
q20 2024 1811 1808 1808
q21 5612 5366 5347 5347
q22 1074 986 1001 986
Total cold run time: 63461 ms
Total hot run time: 58789 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](restore) Disable alter ops during atomic restore [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40817:
URL: https://github.com/apache/doris/pull/40817#issuecomment-2349167786
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org