You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/02/29 09:15:46 UTC
[PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
liaoxin01 opened a new pull request, #31602:
URL: https://github.com/apache/doris/pull/31602
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Check failed: expected_st || st.is() unexpected error status while lookup_row_key:[CORRUPTION]Bad page: too small size (0)
When running this injection case, the statistical information is executed in the background, which may cause the table of statistical information to generate a bad page.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970804625
<details>
<summary>ClickBench: <b>Total hot run time: 31.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0abd85591501e83b78eb298391ecee10d357bff5, data reload: false
query1 0.04 0.04 0.04
query2 0.05 0.02 0.03
query3 0.24 0.07 0.06
query4 1.67 0.10 0.09
query5 0.52 0.51 0.51
query6 1.31 0.68 0.68
query7 0.02 0.02 0.01
query8 0.04 0.03 0.03
query9 0.57 0.52 0.50
query10 0.56 0.56 0.57
query11 0.13 0.10 0.10
query12 0.13 0.11 0.11
query13 0.62 0.61 0.63
query14 0.78 0.80 0.80
query15 0.84 0.82 0.81
query16 0.37 0.36 0.38
query17 1.00 0.99 0.95
query18 0.27 0.25 0.25
query19 1.79 1.70 1.68
query20 0.01 0.01 0.02
query21 15.42 0.68 0.61
query22 2.79 4.38 2.14
query23 17.65 1.11 1.01
query24 2.13 0.37 0.38
query25 0.30 0.05 0.03
query26 0.20 0.15 0.14
query27 0.04 0.03 0.03
query28 12.36 0.84 0.83
query29 12.57 3.20 3.29
query30 0.56 0.54 0.57
query31 2.78 0.34 0.37
query32 3.36 0.48 0.48
query33 3.17 3.21 3.22
query34 15.37 4.76 4.71
query35 4.75 4.76 4.76
query36 1.13 1.06 1.07
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.02 0.02
query40 0.16 0.12 0.13
query41 0.08 0.02 0.02
query42 0.03 0.01 0.02
query43 0.02 0.02 0.03
Total cold run time: 105.96 s
Total hot run time: 31.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970759213
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970719513
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970776912
<details>
<summary>TPC-H: <b>Total hot run time: 38265 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0abd85591501e83b78eb298391ecee10d357bff5, data reload: false
------ Round 1 ----------------------------------
q1 17616 4072 4072 4072
q2 2051 143 143 143
q3 10579 946 964 946
q4 4804 965 982 965
q5 7907 2923 3033 2923
q6 179 125 125 125
q7 1226 755 757 755
q8 9256 2035 2045 2035
q9 7242 6338 6308 6308
q10 8189 2539 2526 2526
q11 416 209 245 209
q12 773 316 307 307
q13 17968 3220 3211 3211
q14 283 256 244 244
q15 535 496 499 496
q16 479 398 429 398
q17 936 900 898 898
q18 6569 6074 6035 6035
q19 1583 1545 1530 1530
q20 552 288 280 280
q21 6604 3576 3576 3576
q22 810 306 283 283
Total cold run time: 106557 ms
Total hot run time: 38265 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4072 4040 4069 4040
q2 303 217 222 217
q3 2991 2912 2948 2912
q4 1856 1839 1819 1819
q5 5150 5105 5159 5105
q6 201 113 116 113
q7 2168 1707 1682 1682
q8 3161 3238 3233 3233
q9 8285 8311 8249 8249
q10 6253 3690 3666 3666
q11 508 423 445 423
q12 679 522 533 522
q13 3794 3091 3070 3070
q14 286 253 263 253
q15 537 485 497 485
q16 509 462 458 458
q17 1718 1687 1691 1687
q18 7735 7521 7551 7521
q19 1699 1655 1650 1650
q20 2142 1925 1899 1899
q21 4768 4666 4609 4609
q22 525 438 430 430
Total cold run time: 59340 ms
Total hot run time: 54043 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970734991
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #31602:
URL: https://github.com/apache/doris/pull/31602
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970759154
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970740673
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970810258
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 0abd85591501e83b78eb298391ecee10d357bff5 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 17.8 seconds inserted 10000000 Rows, about 561K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970719205
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970725273
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970788985
TeamCity be ut coverage result:
Function Coverage: 35.68% (8550/23964)
Line Coverage: 27.49% (69390/252457)
Region Coverage: 26.64% (35983/135077)
Branch Coverage: 23.45% (18389/78434)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0abd85591501e83b78eb298391ecee10d357bff5_0abd85591501e83b78eb298391ecee10d357bff5/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) delete bytes_appended debug point which may cause bad page [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31602:
URL: https://github.com/apache/doris/pull/31602#issuecomment-1970795314
<details>
<summary>TPC-DS: <b>Total hot run time: 168776 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0abd85591501e83b78eb298391ecee10d357bff5, data reload: false
query1 915 342 331 331
query2 6544 1745 1746 1745
query3 6712 218 215 215
query4 23122 20454 20541 20454
query5 4283 363 355 355
query6 252 176 169 169
query7 4605 294 286 286
query8 259 192 211 192
query9 8487 2237 2203 2203
query10 419 215 202 202
query11 14565 14176 14206 14176
query12 134 82 83 82
query13 1642 424 426 424
query14 8369 6820 6568 6568
query15 212 172 176 172
query16 7091 262 256 256
query17 1032 520 494 494
query18 1896 262 259 259
query19 188 145 150 145
query20 90 83 87 83
query21 196 128 129 128
query22 4609 4567 4542 4542
query23 30612 29917 29887 29887
query24 11213 3140 3043 3043
query25 627 356 349 349
query26 1645 152 153 152
query27 2994 307 312 307
query28 6936 1795 1809 1795
query29 1104 547 540 540
query30 283 134 141 134
query31 864 689 690 689
query32 92 61 61 61
query33 713 221 230 221
query34 1051 475 482 475
query35 844 739 735 735
query36 925 887 847 847
query37 129 60 63 60
query38 3072 3001 2986 2986
query39 1335 1239 1244 1239
query40 284 99 100 99
query41 39 37 37 37
query42 107 101 97 97
query43 484 420 421 420
query44 1064 692 703 692
query45 198 184 173 173
query46 1031 788 771 771
query47 1469 1387 1493 1387
query48 418 346 342 342
query49 1131 290 301 290
query50 775 362 376 362
query51 4371 4261 4339 4261
query52 99 95 91 91
query53 337 269 266 266
query54 311 226 229 226
query55 89 77 86 77
query56 222 197 193 193
query57 960 899 827 827
query58 215 198 191 191
query59 2103 2113 2143 2113
query60 236 219 207 207
query61 91 84 84 84
query62 617 327 365 327
query63 285 251 258 251
query64 6618 3026 3137 3026
query65 3235 3222 3208 3208
query66 1431 328 313 313
query67 14560 14164 13817 13817
query68 5018 559 553 553
query69 544 377 372 372
query70 1236 1218 1158 1158
query71 356 250 243 243
query72 6192 2777 2640 2640
query73 722 318 316 316
query74 6288 6068 5992 5992
query75 3043 2479 2449 2449
query76 2832 995 1142 995
query77 349 234 244 234
query78 8898 8790 8416 8416
query79 1090 527 509 509
query80 665 363 361 361
query81 454 199 194 194
query82 1280 94 84 84
query83 238 123 117 117
query84 282 80 78 78
query85 1137 348 334 334
query86 309 303 301 301
query87 3291 3084 3086 3084
query88 2725 2320 2296 2296
query89 391 327 330 327
query90 2009 164 165 164
query91 150 122 122 122
query92 56 48 51 48
query93 1040 521 498 498
query94 1042 185 177 177
query95 439 350 346 346
query96 581 268 264 264
query97 4269 4102 4144 4102
query98 224 207 192 192
query99 1104 661 678 661
Total cold run time: 261095 ms
Total hot run time: 168776 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org