You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "kaka11chen (via GitHub)" <gi...@apache.org> on 2025/12/23 17:17:19 UTC
[PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
kaka11chen opened a new pull request, #59307:
URL: https://github.com/apache/doris/pull/59307
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3749489386
<details>
<summary>ClickBench: <b>Total hot run time: 28.43 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f, data reload: false
query1 0.06 0.05 0.04
query2 0.13 0.07 0.07
query3 0.30 0.08 0.08
query4 1.61 0.11 0.10
query5 0.26 0.23 0.26
query6 1.14 0.65 0.64
query7 0.03 0.03 0.02
query8 0.07 0.06 0.06
query9 0.57 0.50 0.50
query10 0.56 0.56 0.56
query11 0.26 0.13 0.13
query12 0.25 0.14 0.14
query13 0.62 0.60 0.60
query14 0.98 0.97 0.99
query15 0.90 0.81 0.83
query16 0.38 0.39 0.39
query17 0.99 1.06 1.05
query18 0.24 0.22 0.23
query19 1.95 1.87 1.85
query20 0.02 0.01 0.02
query21 15.44 0.27 0.23
query22 5.00 0.10 0.10
query23 15.41 0.39 0.22
query24 2.35 0.50 0.33
query25 0.11 0.09 0.10
query26 0.19 0.17 0.18
query27 0.10 0.09 0.09
query28 3.73 1.12 0.97
query29 12.53 4.10 3.25
query30 0.32 0.14 0.13
query31 2.79 0.70 0.44
query32 3.23 0.61 0.50
query33 3.10 3.02 3.17
query34 16.41 5.06 4.40
query35 4.51 4.96 4.98
query36 0.71 0.54 0.56
query37 0.27 0.08 0.08
query38 0.26 0.05 0.06
query39 0.06 0.06 0.05
query40 0.20 0.18 0.18
query41 0.13 0.07 0.07
query42 0.07 0.04 0.04
query43 0.06 0.06 0.05
Total cold run time: 98.3 s
Total hot run time: 28.43 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752726296
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/7cb3056f1e37e48a937fe3060608b5e62e84c8d5_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/7cb3056f1e37e48a937fe3060608b5e62e84c8d5_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.57% (1784/2242) |
| Line Coverage | 64.78% (31738/48997) |
| Region Coverage | 65.46% (15789/24120) |
| Branch Coverage | 56.02% (8385/14968) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3772708781
# FE UT Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on code in PR #59307:
URL: https://github.com/apache/doris/pull/59307#discussion_r2707968085
##########
be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:
##########
@@ -305,6 +495,32 @@ void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_reserve_decompress_buf(siz
}
}
+template <bool IN_COLLECTION, bool OFFSET_INDEX>
+void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_insert_page_into_cache(
+ const std::vector<uint8_t>& level_bytes, const Slice& payload) {
+ StoragePageCache::CacheKey key(
+ fmt::format("{}::{}", _stream_reader->path(), _stream_reader->mtime()),
+ _page_reader->file_end_offset(), _page_reader->header_start_offset());
+ const std::vector<uint8_t>& header_bytes = _page_reader->header_bytes();
+ size_t total = header_bytes.size() + level_bytes.size() + payload.size;
+ auto* page = new DataPage(total, true, segment_v2::DATA_PAGE);
Review Comment:
No leak here: StoragePageCache::insert takes ownership of the DataPage* and the LRU cache frees it via LRUHandle::free() (which deletes LRUCacheValueBase) when the entry is evicted or the
last handle is released, so the allocation in be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp is managed by the cache (be/src/olap/page_cache.cpp, be/src/olap/
lru_cache.h). Even in the LRU‑K “first insert not cached” path, the returned handle release will free the value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3773042558
<details>
<summary>TPC-DS: <b>Total hot run time: 173930 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c18b9a055f056a4348a0abe92bb480bbfa927357, data reload: false
query5 4483 634 522 522
query6 332 237 218 218
query7 4220 471 264 264
query8 357 266 253 253
query9 8722 2880 2909 2880
query10 505 386 339 339
query11 15180 15124 14764 14764
query12 179 118 117 117
query13 1248 497 411 411
query14 5954 3061 2782 2782
query14_1 2642 2639 2728 2639
query15 208 194 179 179
query16 1003 501 484 484
query17 1095 667 580 580
query18 2426 429 340 340
query19 219 221 198 198
query20 119 114 115 114
query21 215 150 118 118
query22 3838 3935 3803 3803
query23 16103 15645 15321 15321
query23_1 15436 15580 15469 15469
query24 7195 1537 1184 1184
query24_1 1190 1179 1199 1179
query25 553 458 401 401
query26 1259 269 155 155
query27 2761 450 280 280
query28 4551 2187 2182 2182
query29 753 538 430 430
query30 317 241 209 209
query31 816 631 592 592
query32 82 76 77 76
query33 526 373 317 317
query34 909 868 531 531
query35 724 752 689 689
query36 831 902 852 852
query37 139 114 86 86
query38 2807 2747 2656 2656
query39 772 751 748 748
query39_1 725 718 712 712
query40 226 148 129 129
query41 75 67 68 67
query42 108 101 105 101
query43 488 429 415 415
query44 1312 793 745 745
query45 188 190 172 172
query46 845 933 573 573
query47 1424 1455 1291 1291
query48 322 322 231 231
query49 596 425 343 343
query50 622 266 199 199
query51 3723 3791 3685 3685
query52 99 104 93 93
query53 288 324 265 265
query54 280 268 253 253
query55 82 81 74 74
query56 297 300 316 300
query57 1017 1069 950 950
query58 262 257 254 254
query59 2143 2221 2051 2051
query60 337 331 312 312
query61 147 146 149 146
query62 388 347 322 322
query63 290 264 262 262
query64 4901 1263 942 942
query65 3746 3759 3744 3744
query66 1466 428 304 304
query67 15592 15570 15443 15443
query68 2448 1112 783 783
query69 446 361 325 325
query70 993 949 931 931
query71 316 312 289 289
query72 5271 3143 3202 3143
query73 617 729 317 317
query74 8714 8684 8547 8547
query75 2740 2804 2459 2459
query76 2261 1070 651 651
query77 361 379 310 310
query78 9776 10081 9201 9201
query79 1106 920 581 581
query80 1458 586 477 477
query81 548 259 237 237
query82 996 146 111 111
query83 353 254 237 237
query84 250 116 92 92
query85 1189 473 436 436
query86 415 303 291 291
query87 2888 2950 2736 2736
query88 3513 2628 2579 2579
query89 385 342 341 341
query90 1879 177 174 174
query91 162 158 134 134
query92 85 74 65 65
query93 1035 903 548 548
query94 645 328 293 293
query95 581 389 310 310
query96 643 509 233 233
query97 2340 2404 2295 2295
query98 211 202 208 202
query99 632 552 465 465
Total cold run time: 247273 ms
Total hot run time: 173930 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3687396599
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698178200
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c82a283be5705daae1e30b7d0b1d8012e62fca31_c82a283be5705daae1e30b7d0b1d8012e62fca31_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c82a283be5705daae1e30b7d0b1d8012e62fca31_c82a283be5705daae1e30b7d0b1d8012e62fca31_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.52% (1771/2227) |
| Line Coverage | 64.83% (31331/48325) |
| Region Coverage | 65.37% (15584/23839) |
| Branch Coverage | 55.96% (8282/14800) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #59307:
URL: https://github.com/apache/doris/pull/59307
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3773944136
# FE Regression Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689734067
<details>
<summary>TPC-DS: <b>Total hot run time: 179957 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7a15d5f440c5328181e229a72f8fd4a7959ca301, data reload: false
query5 4964 614 462 462
query6 341 232 222 222
query7 4219 468 268 268
query8 300 245 241 241
query9 8765 2544 2549 2544
query10 535 371 319 319
query11 15575 14938 15036 14938
query12 175 119 121 119
query13 1256 484 395 395
query14 6087 2989 2755 2755
query14_1 2621 2612 2617 2612
query15 217 197 179 179
query16 860 483 449 449
query17 1135 726 605 605
query18 2493 447 347 347
query19 254 234 210 210
query20 124 120 119 119
query21 233 143 119 119
query22 3956 4003 4044 4003
query23 16588 16245 16059 16059
query23_1 16415 16044 16150 16044
query24 7445 1656 1210 1210
query24_1 1256 1235 1277 1235
query25 588 513 466 466
query26 1262 277 165 165
query27 2739 474 315 315
query28 4489 2136 2133 2133
query29 834 564 465 465
query30 325 246 217 217
query31 824 726 626 626
query32 81 70 76 70
query33 549 384 313 313
query34 893 902 536 536
query35 782 789 706 706
query36 878 924 804 804
query37 140 104 90 90
query38 3006 3034 2976 2976
query39 750 735 729 729
query39_1 709 740 714 714
query40 232 146 126 126
query41 70 69 70 69
query42 114 111 106 106
query43 412 424 412 412
query44 1363 752 739 739
query45 194 193 194 193
query46 883 999 626 626
query47 1665 1713 1627 1627
query48 326 335 263 263
query49 671 482 368 368
query50 663 305 236 236
query51 3768 3826 3751 3751
query52 107 113 100 100
query53 327 352 294 294
query54 303 275 311 275
query55 79 78 70 70
query56 283 318 308 308
query57 1161 1169 1081 1081
query58 258 256 256 256
query59 2435 2415 2375 2375
query60 308 309 284 284
query61 160 152 156 152
query62 732 715 663 663
query63 321 295 301 295
query64 4948 1304 989 989
query65 4045 3976 3950 3950
query66 1454 440 331 331
query67 15653 15238 15085 15085
query68 8004 1000 716 716
query69 509 350 379 350
query70 1072 995 967 967
query71 373 305 293 293
query72 6063 4956 5104 4956
query73 704 622 314 314
query74 8750 8965 8748 8748
query75 3187 3169 2782 2782
query76 3878 1147 756 756
query77 518 416 297 297
query78 9521 9678 8851 8851
query79 1313 869 606 606
query80 695 633 547 547
query81 514 261 235 235
query82 211 135 103 103
query83 263 257 240 240
query84 253 122 99 99
query85 897 499 468 468
query86 385 296 288 288
query87 3245 3175 3121 3121
query88 3172 2263 2268 2263
query89 459 422 382 382
query90 2211 158 150 150
query91 173 167 143 143
query92 84 71 70 70
query93 1140 910 560 560
query94 490 295 266 266
query95 575 379 307 307
query96 609 459 217 217
query97 2263 2306 2266 2266
query98 221 198 194 194
query99 1307 1328 1342 1328
Total cold run time: 260635 ms
Total hot run time: 179957 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3712912026
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710784452
<details>
<summary>ClickBench: <b>Total hot run time: 27.94 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d038e2ebb872beac2bf3196c7dca50460e9d3d9e, data reload: false
query1 0.05 0.05 0.04
query2 0.13 0.07 0.06
query3 0.32 0.07 0.08
query4 1.61 0.11 0.10
query5 0.27 0.25 0.25
query6 1.14 0.65 0.66
query7 0.03 0.03 0.03
query8 0.07 0.06 0.07
query9 0.58 0.51 0.49
query10 0.56 0.56 0.54
query11 0.26 0.14 0.14
query12 0.25 0.14 0.14
query13 0.64 0.63 0.60
query14 0.99 0.98 0.99
query15 0.88 0.81 0.81
query16 0.38 0.38 0.38
query17 1.00 1.04 1.04
query18 0.26 0.22 0.22
query19 1.94 1.84 1.81
query20 0.02 0.01 0.02
query21 15.39 0.27 0.25
query22 4.95 0.10 0.10
query23 15.41 0.40 0.22
query24 2.40 0.48 0.32
query25 0.10 0.10 0.10
query26 0.18 0.18 0.18
query27 0.09 0.10 0.09
query28 3.77 1.15 0.96
query29 12.58 4.09 3.26
query30 0.35 0.14 0.11
query31 2.81 0.66 0.45
query32 3.27 0.62 0.50
query33 2.98 3.07 3.07
query34 16.95 5.12 4.42
query35 4.59 4.52 4.51
query36 0.61 0.51 0.48
query37 0.25 0.09 0.09
query38 0.21 0.05 0.06
query39 0.07 0.05 0.05
query40 0.20 0.18 0.15
query41 0.14 0.06 0.06
query42 0.08 0.05 0.05
query43 0.06 0.05 0.05
Total cold run time: 98.82 s
Total hot run time: 27.94 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710379186
# FE UT Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_6cffdbf025a1a421ddcec22c261d09dfbfaf71af/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_6cffdbf025a1a421ddcec22c261d09dfbfaf71af/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713049724
# BE UT Coverage Report
Increment line coverage `86.59% (355/410)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/d130bbc50fa98b3b61631d6dd449cd7882bd60f7_d130bbc50fa98b3b61631d6dd449cd7882bd60f7/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/d130bbc50fa98b3b61631d6dd449cd7882bd60f7_d130bbc50fa98b3b61631d6dd449cd7882bd60f7/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.28% (19003/35668) |
| Line Coverage | 39.31% (176556/449158) |
| Region Coverage | 33.81% (136550/403883) |
| Branch Coverage | 34.78% (58933/169469) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3697304198
<details>
<summary>TPC-DS: <b>Total hot run time: 174556 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a8f6d42fbc7af5425b4412fbf5ef84b6851e145b, data reload: false
query5 4580 585 433 433
query6 335 226 208 208
query7 4225 459 284 284
query8 353 258 245 245
query9 8800 2617 2607 2607
query10 539 380 314 314
query11 15445 15131 14784 14784
query12 190 117 120 117
query13 1268 520 389 389
query14 6221 3021 2660 2660
query14_1 2583 2574 2586 2574
query15 205 206 179 179
query16 990 465 445 445
query17 1076 675 560 560
query18 2526 430 335 335
query19 218 216 197 197
query20 124 119 120 119
query21 226 141 127 127
query22 3785 3922 4134 3922
query23 16074 15516 15308 15308
query23_1 15412 15406 15383 15383
query24 7345 1603 1180 1180
query24_1 1195 1205 1210 1205
query25 552 466 443 443
query26 1242 267 166 166
query27 2768 463 303 303
query28 4517 2203 2177 2177
query29 783 555 448 448
query30 318 243 215 215
query31 807 637 571 571
query32 82 73 67 67
query33 546 335 296 296
query34 907 873 526 526
query35 769 782 685 685
query36 857 875 812 812
query37 127 97 90 90
query38 2703 2747 2637 2637
query39 778 762 726 726
query39_1 722 718 723 718
query40 231 135 120 120
query41 74 68 66 66
query42 108 113 111 111
query43 441 420 423 420
query44 1361 755 755 755
query45 187 188 175 175
query46 866 981 616 616
query47 1398 1448 1386 1386
query48 319 338 279 279
query49 621 431 337 337
query50 639 296 211 211
query51 3800 3832 3834 3832
query52 107 109 100 100
query53 287 326 269 269
query54 298 279 260 260
query55 82 79 74 74
query56 295 297 336 297
query57 1037 984 903 903
query58 286 263 306 263
query59 2034 2190 2009 2009
query60 329 310 293 293
query61 167 164 168 164
query62 395 360 332 332
query63 292 258 268 258
query64 4840 1290 1011 1011
query65 3828 3721 3681 3681
query66 1417 438 313 313
query67 15182 14925 15406 14925
query68 7494 1010 725 725
query69 500 344 303 303
query70 1077 960 916 916
query71 370 293 276 276
query72 6254 4849 4961 4849
query73 696 634 304 304
query74 8814 8773 8538 8538
query75 2933 2898 2535 2535
query76 3880 1052 671 671
query77 531 374 289 289
query78 9857 9772 9126 9126
query79 1924 874 599 599
query80 691 572 498 498
query81 519 260 233 233
query82 471 140 111 111
query83 266 249 231 231
query84 267 120 105 105
query85 987 517 465 465
query86 392 293 316 293
query87 2934 2890 2712 2712
query88 4427 2274 2268 2268
query89 389 359 329 329
query90 2127 161 155 155
query91 179 170 144 144
query92 76 67 66 66
query93 2224 901 560 560
query94 577 321 294 294
query95 579 328 361 328
query96 589 505 204 204
query97 2349 2362 2342 2342
query98 236 204 200 200
query99 621 586 516 516
Total cold run time: 257371 ms
Total hot run time: 174556 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3692751965
# BE UT Coverage Report
Increment line coverage `91.60% (240/262)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45_3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45_3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.43% (18968/35502) |
| Line Coverage | 39.30% (176038/447932) |
| Region Coverage | 33.86% (136182/402158) |
| Branch Coverage | 34.79% (58799/169032) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3748990229
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3773475863
# BE UT Coverage Report
Increment line coverage `81.55% (336/412)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c18b9a055f056a4348a0abe92bb480bbfa927357_c18b9a055f056a4348a0abe92bb480bbfa927357/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c18b9a055f056a4348a0abe92bb480bbfa927357_c18b9a055f056a4348a0abe92bb480bbfa927357/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.42% (19070/36379) |
| Line Coverage | 35.73% (176672/494425) |
| Region Coverage | 32.27% (136767/423885) |
| Branch Coverage | 33.16% (59126/178312) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on code in PR #59307:
URL: https://github.com/apache/doris/pull/59307#discussion_r2707877681
##########
be/src/vec/exec/format/parquet/vparquet_page_reader.cpp:
##########
@@ -77,11 +83,79 @@ Status PageReader<IN_COLLECTION, OFFSET_INDEX>::parse_page_header() {
return Status::IOError("Should skip or load current page to get next page");
}
+ _page_statistics.page_read_counter += 1;
+
+ // Parse page header from file; header bytes are saved for possible cache insertion
const uint8_t* page_header_buf = nullptr;
size_t max_size = _end_offset - _offset;
size_t header_size = std::min(INIT_PAGE_HEADER_SIZE, max_size);
const size_t MAX_PAGE_HEADER_SIZE = config::parquet_header_max_size_mb << 20;
uint32_t real_header_size = 0;
+
+ // Try a header-only lookup in the page cache. Cached pages store
+ // header + optional v2 levels + uncompressed payload, so we can
+ // parse the page header directly from the cached bytes and avoid
+ // a file read for the header.
+ if (_page_read_ctx.enable_parquet_file_page_cache && !config::disable_storage_page_cache &&
+ StoragePageCache::instance() != nullptr) {
+ PageCacheHandle handle;
+ StoragePageCache::CacheKey key(fmt::format("{}::{}", _reader->path(), _reader->mtime()),
+ _end_offset, _offset);
+ if (StoragePageCache::instance()->lookup(key, &handle, segment_v2::DATA_PAGE)) {
+ // Parse header directly from cached data
+ _page_cache_handle = std::move(handle);
+ Slice s = _page_cache_handle.data();
+ real_header_size = cast_set<uint32_t>(s.size);
+ SCOPED_RAW_TIMER(&_page_statistics.decode_header_time);
+ auto st = deserialize_thrift_msg(reinterpret_cast<const uint8_t*>(s.data),
+ &real_header_size, true, &_cur_page_header);
+ if (!st.ok()) return st;
+ // Increment page cache counters for a true cache hit on header+payload
+ _page_statistics.page_cache_hit_counter += 1;
+ // Detect whether the cached payload is compressed or decompressed and record
+ bool is_cache_payload_decompressed = true;
+ if (_cur_page_header.compressed_page_size > 0) {
Review Comment:
ok
##########
be/src/io/file_factory.cpp:
##########
@@ -203,6 +203,21 @@ Result<io::FileReaderSPtr> FileFactory::create_file_reader(
const io::FileSystemProperties& system_properties,
const io::FileDescription& file_description, const io::FileReaderOptions& reader_options,
RuntimeProfile* profile) {
+ auto reader_res = _create_file_reader_internal(system_properties, file_description,
+ reader_options, profile);
+ if (!reader_res.has_value()) {
+ return unexpected(std::move(reader_res).error());
+ }
+ auto file_reader = std::move(reader_res).value();
+ LOG_INFO("create file reader for path={}, size={}, mtime={}", file_description.path,
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710252960
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6cffdbf025a1a421ddcec22c261d09dfbfaf71af_6cffdbf025a1a421ddcec22c261d09dfbfaf71af_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6cffdbf025a1a421ddcec22c261d09dfbfaf71af_6cffdbf025a1a421ddcec22c261d09dfbfaf71af_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.53% (1776/2233) |
| Line Coverage | 64.82% (31443/48508) |
| Region Coverage | 65.39% (15640/23918) |
| Branch Coverage | 56.03% (8323/14854) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3714438708
# FE Regression Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_b119cd5b30259d7c46ff99fdec66478a31180761_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_b119cd5b30259d7c46ff99fdec66478a31180761_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710763356
<details>
<summary>TPC-DS: <b>Total hot run time: 172478 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d038e2ebb872beac2bf3196c7dca50460e9d3d9e, data reload: false
query5 4442 585 445 445
query6 340 239 222 222
query7 4217 452 263 263
query8 340 249 248 248
query9 8735 2621 2656 2621
query10 528 369 329 329
query11 15081 15083 14795 14795
query12 174 119 115 115
query13 1262 500 377 377
query14 6092 2992 2749 2749
query14_1 2658 2636 2666 2636
query15 203 196 178 178
query16 1028 478 446 446
query17 1112 692 580 580
query18 2477 438 348 348
query19 229 227 201 201
query20 122 119 118 118
query21 213 141 117 117
query22 4073 3934 3789 3789
query23 15866 15460 15327 15327
query23_1 15555 15353 15322 15322
query24 7180 1511 1149 1149
query24_1 1175 1158 1170 1158
query25 564 476 429 429
query26 1263 269 162 162
query27 2744 448 290 290
query28 4429 2127 2127 2127
query29 787 561 476 476
query30 312 243 215 215
query31 858 625 545 545
query32 73 71 70 70
query33 528 324 274 274
query34 880 864 524 524
query35 775 767 708 708
query36 849 862 766 766
query37 126 90 80 80
query38 2764 2720 2671 2671
query39 782 756 714 714
query39_1 727 723 695 695
query40 218 126 110 110
query41 64 77 74 74
query42 107 102 101 101
query43 421 465 402 402
query44 1292 720 718 718
query45 188 182 173 173
query46 838 941 584 584
query47 1423 1368 1400 1368
query48 309 314 236 236
query49 621 397 329 329
query50 619 271 200 200
query51 3763 3872 3731 3731
query52 99 104 95 95
query53 290 321 269 269
query54 283 247 244 244
query55 74 74 70 70
query56 273 280 298 280
query57 1000 969 927 927
query58 273 250 249 249
query59 2028 2149 2037 2037
query60 320 306 299 299
query61 158 150 188 150
query62 391 346 344 344
query63 298 266 267 266
query64 4758 1284 981 981
query65 3708 3739 3718 3718
query66 1419 435 302 302
query67 15239 15237 15284 15237
query68 8038 971 693 693
query69 506 371 317 317
query70 1058 930 920 920
query71 370 312 278 278
query72 6121 3398 3427 3398
query73 769 707 302 302
query74 8769 8786 8589 8589
query75 2785 2865 2437 2437
query76 3930 1051 644 644
query77 533 385 278 278
query78 9600 9666 9088 9088
query79 1274 896 603 603
query80 627 551 484 484
query81 512 256 231 231
query82 209 142 112 112
query83 260 263 241 241
query84 261 128 99 99
query85 876 498 465 465
query86 396 298 320 298
query87 2835 2922 2787 2787
query88 3258 2244 2234 2234
query89 382 350 330 330
query90 2176 157 154 154
query91 168 164 139 139
query92 86 73 63 63
query93 1814 903 527 527
query94 569 320 288 288
query95 570 377 310 310
query96 588 474 208 208
query97 2282 2355 2300 2300
query98 239 196 195 195
query99 574 597 496 496
Total cold run time: 253520 ms
Total hot run time: 172478 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713050269
<details>
<summary>ClickBench: <b>Total hot run time: 27.87 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d130bbc50fa98b3b61631d6dd449cd7882bd60f7, data reload: false
query1 0.05 0.05 0.05
query2 0.13 0.07 0.07
query3 0.32 0.08 0.07
query4 1.60 0.10 0.09
query5 0.26 0.24 0.25
query6 1.14 0.65 0.67
query7 0.03 0.03 0.03
query8 0.07 0.06 0.06
query9 0.58 0.52 0.51
query10 0.55 0.55 0.55
query11 0.26 0.13 0.14
query12 0.26 0.14 0.14
query13 0.63 0.61 0.61
query14 0.99 0.98 0.98
query15 0.88 0.80 0.80
query16 0.40 0.38 0.40
query17 1.03 0.99 1.02
query18 0.24 0.22 0.22
query19 1.94 1.81 1.78
query20 0.02 0.01 0.02
query21 15.41 0.28 0.23
query22 4.99 0.09 0.10
query23 15.41 0.38 0.23
query24 2.40 0.45 0.32
query25 0.10 0.10 0.09
query26 0.17 0.17 0.16
query27 0.10 0.09 0.09
query28 3.69 1.15 0.96
query29 12.55 4.02 3.28
query30 0.32 0.13 0.10
query31 2.81 0.67 0.43
query32 3.24 0.62 0.49
query33 2.95 3.07 3.06
query34 17.06 5.10 4.50
query35 4.54 4.56 4.48
query36 0.62 0.50 0.48
query37 0.24 0.08 0.08
query38 0.20 0.06 0.06
query39 0.07 0.05 0.05
query40 0.18 0.17 0.17
query41 0.13 0.07 0.06
query42 0.07 0.05 0.05
query43 0.06 0.06 0.05
Total cold run time: 98.69 s
Total hot run time: 27.87 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3772548909
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c18b9a055f056a4348a0abe92bb480bbfa927357_c18b9a055f056a4348a0abe92bb480bbfa927357_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c18b9a055f056a4348a0abe92bb480bbfa927357_c18b9a055f056a4348a0abe92bb480bbfa927357_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.52% (1786/2246) |
| Line Coverage | 64.75% (31802/49114) |
| Region Coverage | 65.48% (15846/24201) |
| Branch Coverage | 56.01% (8417/15028) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3753261577
# FE Regression Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752982280
<details>
<summary>TPC-DS: <b>Total hot run time: 174421 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7cb3056f1e37e48a937fe3060608b5e62e84c8d5, data reload: false
query5 4458 626 497 497
query6 358 237 238 237
query7 4225 464 262 262
query8 347 256 250 250
query9 8704 2861 2882 2861
query10 522 395 359 359
query11 15392 15139 14997 14997
query12 176 118 122 118
query13 1281 501 382 382
query14 6289 3053 2795 2795
query14_1 2712 2763 2673 2673
query15 204 192 170 170
query16 1025 466 476 466
query17 1120 672 591 591
query18 2463 443 340 340
query19 255 226 198 198
query20 132 119 120 119
query21 217 137 122 122
query22 4212 4317 4200 4200
query23 16146 16085 15323 15323
query23_1 15478 15427 15416 15416
query24 7168 1573 1167 1167
query24_1 1172 1200 1183 1183
query25 557 484 429 429
query26 1257 270 155 155
query27 2759 461 288 288
query28 4516 2117 2118 2117
query29 851 516 420 420
query30 312 242 208 208
query31 805 624 580 580
query32 85 73 71 71
query33 532 345 314 314
query34 919 869 533 533
query35 730 764 703 703
query36 868 909 807 807
query37 135 91 86 86
query38 2803 2668 2601 2601
query39 805 739 742 739
query39_1 711 719 690 690
query40 220 139 120 120
query41 67 65 62 62
query42 106 102 101 101
query43 413 476 423 423
query44 1337 731 765 731
query45 182 187 177 177
query46 858 959 576 576
query47 1416 1504 1412 1412
query48 312 321 235 235
query49 600 420 358 358
query50 612 263 205 205
query51 3796 3830 3757 3757
query52 104 103 97 97
query53 290 318 269 269
query54 288 272 259 259
query55 86 85 76 76
query56 311 318 307 307
query57 1049 1043 948 948
query58 269 260 258 258
query59 2164 2163 2054 2054
query60 337 342 321 321
query61 152 149 159 149
query62 387 342 316 316
query63 292 266 262 262
query64 4967 1290 1000 1000
query65 3823 3792 3714 3714
query66 1461 410 302 302
query67 15496 15497 15356 15356
query68 2363 1109 751 751
query69 449 361 330 330
query70 1011 956 958 956
query71 321 312 289 289
query72 5368 3129 3352 3129
query73 609 720 309 309
query74 8686 8713 8500 8500
query75 2714 2792 2469 2469
query76 2288 1062 658 658
query77 361 371 312 312
query78 9717 9906 9160 9160
query79 2308 914 580 580
query80 1748 581 488 488
query81 548 260 234 234
query82 1016 142 112 112
query83 323 253 234 234
query84 258 104 98 98
query85 906 508 441 441
query86 439 300 294 294
query87 2881 2860 2741 2741
query88 3489 2589 2568 2568
query89 402 360 343 343
query90 1998 191 179 179
query91 172 161 135 135
query92 77 75 73 73
query93 1106 885 528 528
query94 648 326 293 293
query95 584 344 385 344
query96 634 498 235 235
query97 2331 2396 2271 2271
query98 215 199 205 199
query99 606 587 524 524
Total cold run time: 249565 ms
Total hot run time: 174421 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713041426
<details>
<summary>TPC-DS: <b>Total hot run time: 172549 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d130bbc50fa98b3b61631d6dd449cd7882bd60f7, data reload: false
query5 4472 563 429 429
query6 333 225 250 225
query7 4210 465 260 260
query8 340 251 264 251
query9 8790 2658 2631 2631
query10 546 371 325 325
query11 15158 15064 14856 14856
query12 179 117 117 117
query13 1272 492 373 373
query14 6247 3000 2727 2727
query14_1 2677 2603 2624 2603
query15 200 193 175 175
query16 995 486 453 453
query17 1084 655 552 552
query18 2570 427 333 333
query19 213 218 191 191
query20 121 118 113 113
query21 210 151 129 129
query22 4090 3936 3977 3936
query23 16018 15685 15371 15371
query23_1 15470 15464 15386 15386
query24 7082 1537 1181 1181
query24_1 1175 1157 1169 1157
query25 530 426 417 417
query26 1252 268 160 160
query27 2759 454 291 291
query28 4435 2141 2133 2133
query29 811 563 465 465
query30 312 250 203 203
query31 776 619 542 542
query32 88 75 69 69
query33 544 356 302 302
query34 880 891 523 523
query35 785 799 726 726
query36 865 860 795 795
query37 128 89 78 78
query38 2797 2740 2735 2735
query39 786 751 719 719
query39_1 725 710 730 710
query40 225 136 119 119
query41 73 71 68 68
query42 107 105 101 101
query43 452 485 421 421
query44 1296 723 723 723
query45 190 188 178 178
query46 858 975 577 577
query47 1474 1419 1393 1393
query48 322 328 255 255
query49 619 446 340 340
query50 655 272 232 232
query51 3799 3791 3728 3728
query52 115 109 97 97
query53 295 328 275 275
query54 306 268 260 260
query55 79 78 76 76
query56 307 299 302 299
query57 1027 1042 957 957
query58 298 257 267 257
query59 2156 2224 2148 2148
query60 334 317 312 312
query61 196 196 230 196
query62 404 368 319 319
query63 292 260 267 260
query64 4740 1314 1009 1009
query65 3768 3539 3698 3539
query66 1385 413 288 288
query67 14791 15598 14948 14948
query68 8200 977 708 708
query69 486 345 314 314
query70 1066 947 939 939
query71 374 290 329 290
query72 6119 3353 3478 3353
query73 745 724 290 290
query74 8834 8846 8559 8559
query75 2838 2812 2451 2451
query76 3963 1049 629 629
query77 535 368 271 271
query78 9665 9941 9190 9190
query79 1266 841 582 582
query80 604 574 454 454
query81 507 256 228 228
query82 207 144 105 105
query83 256 259 242 242
query84 256 122 103 103
query85 887 523 449 449
query86 386 325 324 324
query87 2805 2801 2717 2717
query88 3088 2215 2195 2195
query89 386 347 310 310
query90 2152 157 150 150
query91 175 166 145 145
query92 84 66 61 61
query93 1252 924 534 534
query94 562 316 309 309
query95 564 381 308 308
query96 586 442 207 207
query97 2341 2386 2279 2279
query98 206 202 199 199
query99 610 571 518 518
Total cold run time: 253621 ms
Total hot run time: 172549 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3690091471
# BE Regression && UT Coverage Report
Increment line coverage `95.58% (238/249)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_7a15d5f440c5328181e229a72f8fd4a7959ca301_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_7a15d5f440c5328181e229a72f8fd4a7959ca301_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.27% (25080/34705) |
| Line Coverage | 58.95% (263296/446628) |
| Region Coverage | 53.74% (218251/406124) |
| Branch Coverage | 55.37% (93864/169532) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689808063
# BE UT Coverage Report
Increment line coverage `91.97% (229/249)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/7a15d5f440c5328181e229a72f8fd4a7959ca301_7a15d5f440c5328181e229a72f8fd4a7959ca301/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/7a15d5f440c5328181e229a72f8fd4a7959ca301_7a15d5f440c5328181e229a72f8fd4a7959ca301/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.40% (18958/35501) |
| Line Coverage | 39.26% (175792/447803) |
| Region Coverage | 33.79% (135805/401928) |
| Branch Coverage | 34.71% (58651/168955) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698193114
<details>
<summary>TPC-H: <b>Total hot run time: 34266 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c82a283be5705daae1e30b7d0b1d8012e62fca31, data reload: false
------ Round 1 ----------------------------------
q1 16431 4285 4048 4048
q2 2072 345 238 238
q3 9744 1286 753 753
q4 9798 861 319 319
q5 7357 2145 1856 1856
q6 189 167 136 136
q7 937 805 665 665
q8 9079 1433 1082 1082
q9 6679 5135 5098 5098
q10 6744 1799 1409 1409
q11 495 324 288 288
q12 707 727 571 571
q13 17385 3810 3117 3117
q14 294 292 291 291
q15 577 511 504 504
q16 691 664 628 628
q17 699 748 600 600
q18 7539 7328 7255 7255
q19 929 963 622 622
q20 400 364 255 255
q21 4154 3895 3565 3565
q22 1062 1008 966 966
Total cold run time: 103962 ms
Total hot run time: 34266 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4058 4070 4038 4038
q2 313 397 319 319
q3 2100 2655 2276 2276
q4 1279 1730 1334 1334
q5 4119 4052 4104 4052
q6 267 235 171 171
q7 2138 1997 1782 1782
q8 2588 2441 2369 2369
q9 7139 7092 7147 7092
q10 2487 2848 2250 2250
q11 568 487 459 459
q12 710 797 658 658
q13 3582 4113 3408 3408
q14 272 295 284 284
q15 547 524 509 509
q16 646 672 611 611
q17 1149 1393 1391 1391
q18 7997 7567 7879 7567
q19 902 933 916 916
q20 2045 2045 1934 1934
q21 4770 4354 4087 4087
q22 1083 1033 992 992
Total cold run time: 50759 ms
Total hot run time: 48499 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717530316
# BE UT Coverage Report
Increment line coverage `82.52% (354/429)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/797a4d615363ccd28721ca73fb3a2cd51941185b_797a4d615363ccd28721ca73fb3a2cd51941185b/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/797a4d615363ccd28721ca73fb3a2cd51941185b_797a4d615363ccd28721ca73fb3a2cd51941185b/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.25% (18999/35677) |
| Line Coverage | 39.23% (176080/448789) |
| Region Coverage | 33.79% (136411/403715) |
| Branch Coverage | 34.75% (58840/169347) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3714048840
# BE UT Coverage Report
Increment line coverage `86.87% (364/419)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b119cd5b30259d7c46ff99fdec66478a31180761_b119cd5b30259d7c46ff99fdec66478a31180761/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b119cd5b30259d7c46ff99fdec66478a31180761_b119cd5b30259d7c46ff99fdec66478a31180761/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.25% (18994/35672) |
| Line Coverage | 39.24% (176139/448867) |
| Region Coverage | 33.81% (136540/403892) |
| Branch Coverage | 34.75% (58888/169473) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717931454
# BE Regression && UT Coverage Report
Increment line coverage `86.45% (370/428)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.85% (25756/34875) |
| Line Coverage | 61.21% (273960/447561) |
| Region Coverage | 55.87% (227841/407826) |
| Branch Coverage | 57.90% (98366/169893) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717982176
# FE Regression Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3687739913
<details>
<summary>TPC-DS: <b>Total hot run time: 179978 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e32d9d4931d69e255e597e9d8c195bdeddb732d2, data reload: false
query5 4822 588 426 426
query6 323 232 212 212
query7 4222 456 274 274
query8 313 250 236 236
query9 8787 2562 2535 2535
query10 528 372 318 318
query11 15455 15488 15120 15120
query12 178 114 128 114
query13 1255 492 388 388
query14 6315 3042 2791 2791
query14_1 2660 2625 2755 2625
query15 204 195 173 173
query16 880 480 453 453
query17 1116 705 606 606
query18 2557 430 330 330
query19 229 226 196 196
query20 131 119 115 115
query21 222 137 116 116
query22 3949 4014 3896 3896
query23 16570 16482 15870 15870
query23_1 16113 16163 16008 16008
query24 7341 1644 1205 1205
query24_1 1233 1207 1227 1207
query25 588 468 420 420
query26 1254 274 160 160
query27 2766 463 305 305
query28 4509 2118 2105 2105
query29 786 560 440 440
query30 318 240 208 208
query31 845 730 616 616
query32 76 71 65 65
query33 551 334 292 292
query34 891 883 536 536
query35 779 787 707 707
query36 863 908 812 812
query37 134 99 84 84
query38 2996 3065 3020 3020
query39 770 762 720 720
query39_1 694 703 696 696
query40 229 145 132 132
query41 86 72 64 64
query42 109 108 111 108
query43 431 425 397 397
query44 1299 749 743 743
query45 190 194 185 185
query46 885 992 626 626
query47 1691 1713 1620 1620
query48 315 332 257 257
query49 636 450 373 373
query50 658 309 234 234
query51 3831 3875 3841 3841
query52 112 116 109 109
query53 315 372 306 306
query54 308 279 284 279
query55 87 81 76 76
query56 309 322 326 322
query57 1164 1180 1115 1115
query58 290 276 267 267
query59 2396 2500 2422 2422
query60 332 332 321 321
query61 200 196 188 188
query62 751 744 746 744
query63 335 298 306 298
query64 5066 1412 1126 1126
query65 4028 3979 4003 3979
query66 1450 450 344 344
query67 15264 14989 14849 14849
query68 5458 1016 728 728
query69 510 366 320 320
query70 1091 1026 1009 1009
query71 367 312 310 310
query72 6137 4970 4954 4954
query73 682 581 303 303
query74 8861 8898 8660 8660
query75 3219 3164 2811 2811
query76 3784 1141 749 749
query77 524 381 286 286
query78 9573 9530 8874 8874
query79 1550 942 629 629
query80 730 664 568 568
query81 523 264 232 232
query82 216 133 105 105
query83 258 252 234 234
query84 263 129 106 106
query85 918 514 459 459
query86 388 313 294 294
query87 3244 3176 3150 3150
query88 3300 2267 2260 2260
query89 465 427 402 402
query90 2176 162 153 153
query91 178 175 149 149
query92 86 68 63 63
query93 1775 947 580 580
query94 481 295 285 285
query95 583 328 363 328
query96 602 491 211 211
query97 2258 2302 2220 2220
query98 212 201 191 191
query99 1318 1371 1250 1250
Total cold run time: 258992 ms
Total hot run time: 179978 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689179504
<details>
<summary>ClickBench: <b>Total hot run time: 27.98 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b25ce38cc5917e7ea076b672f14333bd0b196779, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.05
query3 0.26 0.09 0.09
query4 1.60 0.11 0.11
query5 0.27 0.26 0.27
query6 1.18 0.64 0.63
query7 0.03 0.02 0.02
query8 0.05 0.04 0.05
query9 0.57 0.51 0.50
query10 0.56 0.55 0.55
query11 0.15 0.10 0.11
query12 0.15 0.11 0.12
query13 0.61 0.60 0.60
query14 1.00 1.00 0.98
query15 0.81 0.80 0.80
query16 0.40 0.38 0.38
query17 1.04 1.05 1.04
query18 0.23 0.22 0.21
query19 1.86 1.85 1.86
query20 0.02 0.01 0.01
query21 15.47 0.28 0.13
query22 4.85 0.05 0.05
query23 15.93 0.28 0.10
query24 1.46 1.03 0.94
query25 0.11 0.11 0.09
query26 0.14 0.13 0.14
query27 0.07 0.06 0.05
query28 5.87 1.23 1.03
query29 12.62 4.05 3.22
query30 0.29 0.14 0.11
query31 2.84 0.62 0.38
query32 3.22 0.54 0.44
query33 3.01 2.97 2.99
query34 16.96 5.19 4.57
query35 4.61 4.60 4.57
query36 0.66 0.49 0.50
query37 0.11 0.07 0.06
query38 0.08 0.04 0.04
query39 0.04 0.03 0.02
query40 0.16 0.15 0.13
query41 0.08 0.03 0.03
query42 0.04 0.02 0.02
query43 0.04 0.03 0.04
Total cold run time: 99.61 s
Total hot run time: 27.98 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752638422
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3753228293
# BE Regression && UT Coverage Report
Increment line coverage `86.40% (362/419)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_7cb3056f1e37e48a937fe3060608b5e62e84c8d5_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.94% (25990/35152) |
| Line Coverage | 61.40% (276633/450537) |
| Region Coverage | 56.11% (230114/410139) |
| Branch Coverage | 58.11% (99286/170863) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3748952398
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3749310740
# FE UT Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3691602711
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710137881
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3711050328
# BE UT Coverage Report
Increment line coverage `87.06% (350/402)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/d038e2ebb872beac2bf3196c7dca50460e9d3d9e_d038e2ebb872beac2bf3196c7dca50460e9d3d9e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/d038e2ebb872beac2bf3196c7dca50460e9d3d9e_d038e2ebb872beac2bf3196c7dca50460e9d3d9e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.22% (18984/35668) |
| Line Coverage | 39.24% (176265/449150) |
| Region Coverage | 33.74% (136282/403867) |
| Branch Coverage | 34.71% (58815/169456) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713023200
<details>
<summary>TPC-H: <b>Total hot run time: 32198 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d130bbc50fa98b3b61631d6dd449cd7882bd60f7, data reload: false
------ Round 1 ----------------------------------
q1 16444 4300 4085 4085
q2 2020 357 249 249
q3 9750 1258 726 726
q4 9873 883 313 313
q5 7453 2157 1939 1939
q6 193 184 141 141
q7 937 821 674 674
q8 9180 1359 1157 1157
q9 4853 4715 4560 4560
q10 6763 1805 1428 1428
q11 553 303 289 289
q12 712 749 598 598
q13 17579 3819 3189 3189
q14 302 290 272 272
q15 557 516 511 511
q16 706 683 672 672
q17 647 859 489 489
q18 6750 6365 6384 6365
q19 1223 1033 669 669
q20 466 407 292 292
q21 3399 2631 2501 2501
q22 1142 1112 1079 1079
Total cold run time: 101502 ms
Total hot run time: 32198 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4351 4215 4266 4215
q2 334 421 329 329
q3 2296 2819 2468 2468
q4 1466 1816 1412 1412
q5 4404 4506 4347 4347
q6 229 175 126 126
q7 1989 1952 1809 1809
q8 2511 2447 2393 2393
q9 7186 7080 7086 7080
q10 2435 2791 2424 2424
q11 541 480 453 453
q12 703 740 586 586
q13 3573 3819 3101 3101
q14 272 280 260 260
q15 513 485 482 482
q16 612 634 595 595
q17 1094 1242 1287 1242
q18 7233 7334 7121 7121
q19 801 777 798 777
q20 1894 2000 1808 1808
q21 4598 4258 4130 4130
q22 1088 1005 990 990
Total cold run time: 50123 ms
Total hot run time: 48148 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3694876869
<details>
<summary>TPC-H: <b>Total hot run time: 35311 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 59ebab3389bf5deaea3a89564cce010b55705628, data reload: false
------ Round 1 ----------------------------------
q1 17599 4239 4090 4090
q2 2007 361 234 234
q3 10183 1356 751 751
q4 10200 839 317 317
q5 7511 2141 1984 1984
q6 194 178 140 140
q7 1006 880 713 713
q8 9369 1431 1194 1194
q9 7417 5340 5278 5278
q10 6876 2404 1998 1998
q11 541 336 311 311
q12 735 753 572 572
q13 17813 3718 3064 3064
q14 289 298 266 266
q15 603 515 505 505
q16 695 673 632 632
q17 700 822 541 541
q18 7413 7053 6872 6872
q19 1102 979 623 623
q20 405 369 258 258
q21 4341 4075 4000 4000
q22 1042 990 968 968
Total cold run time: 108041 ms
Total hot run time: 35311 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4139 4106 4115 4106
q2 329 406 309 309
q3 2179 2691 2286 2286
q4 1365 1766 1295 1295
q5 4235 4672 4722 4672
q6 238 183 141 141
q7 2074 1938 1819 1819
q8 2731 2604 2577 2577
q9 7442 7553 7432 7432
q10 3134 3248 2837 2837
q11 632 538 541 538
q12 1006 792 587 587
q13 3533 3975 3468 3468
q14 294 309 290 290
q15 570 530 567 530
q16 832 695 634 634
q17 1229 1563 1419 1419
q18 7880 7694 7554 7554
q19 881 837 874 837
q20 1989 2055 1919 1919
q21 4827 4269 4201 4201
q22 1153 1026 983 983
Total cold run time: 52692 ms
Total hot run time: 50434 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3697280243
<details>
<summary>TPC-H: <b>Total hot run time: 35583 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a8f6d42fbc7af5425b4412fbf5ef84b6851e145b, data reload: false
------ Round 1 ----------------------------------
q1 17423 4241 4080 4080
q2 2060 363 243 243
q3 10074 1307 756 756
q4 10217 867 317 317
q5 7540 2253 1907 1907
q6 199 178 137 137
q7 931 806 665 665
q8 9292 1474 1216 1216
q9 6920 5230 5199 5199
q10 6831 1823 1408 1408
q11 516 324 297 297
q12 711 728 604 604
q13 17825 3829 3114 3114
q14 298 310 272 272
q15 603 509 502 502
q16 701 696 653 653
q17 705 736 615 615
q18 7525 7515 8203 7515
q19 1104 1029 640 640
q20 442 426 254 254
q21 4581 4382 4142 4142
q22 1175 1062 1047 1047
Total cold run time: 107673 ms
Total hot run time: 35583 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4274 4224 4220 4220
q2 314 405 313 313
q3 2277 2817 2471 2471
q4 1476 1898 1470 1470
q5 4471 4554 4290 4290
q6 218 165 123 123
q7 2001 1932 1784 1784
q8 2514 2276 2380 2276
q9 7292 7054 7161 7054
q10 2450 2713 2247 2247
q11 541 461 429 429
q12 660 694 571 571
q13 3315 3758 3055 3055
q14 280 278 261 261
q15 522 489 497 489
q16 603 655 608 608
q17 1069 1223 1263 1223
q18 7370 7123 7304 7123
q19 856 834 858 834
q20 1854 1958 1812 1812
q21 4502 4158 4135 4135
q22 1110 1057 977 977
Total cold run time: 49969 ms
Total hot run time: 47765 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698204391
<details>
<summary>TPC-DS: <b>Total hot run time: 174093 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c82a283be5705daae1e30b7d0b1d8012e62fca31, data reload: false
query5 4428 594 452 452
query6 320 229 241 229
query7 4237 460 272 272
query8 333 258 243 243
query9 8790 2609 2627 2609
query10 520 383 310 310
query11 15378 15103 14775 14775
query12 175 116 115 115
query13 1265 481 382 382
query14 5749 3038 2678 2678
query14_1 2643 2547 2570 2547
query15 199 196 175 175
query16 1016 470 454 454
query17 1077 653 556 556
query18 2432 428 326 326
query19 222 216 196 196
query20 119 112 116 112
query21 211 147 119 119
query22 4037 3962 3894 3894
query23 15993 15557 15395 15395
query23_1 15556 15352 15364 15352
query24 7086 1578 1169 1169
query24_1 1210 1197 1196 1196
query25 524 461 436 436
query26 1252 264 165 165
query27 2745 459 296 296
query28 4379 2173 2169 2169
query29 787 541 456 456
query30 314 242 214 214
query31 786 635 546 546
query32 79 69 70 69
query33 553 349 297 297
query34 894 883 532 532
query35 756 781 692 692
query36 847 894 820 820
query37 125 88 77 77
query38 2703 2720 2686 2686
query39 777 759 727 727
query39_1 703 724 702 702
query40 221 132 122 122
query41 89 67 67 67
query42 113 108 102 102
query43 429 464 404 404
query44 1355 773 749 749
query45 194 179 180 179
query46 865 972 610 610
query47 1442 1406 1330 1330
query48 320 337 244 244
query49 622 435 341 341
query50 650 281 209 209
query51 3741 3850 3781 3781
query52 108 111 97 97
query53 294 325 271 271
query54 303 275 267 267
query55 81 79 73 73
query56 293 296 320 296
query57 1000 968 945 945
query58 283 257 252 252
query59 2193 2082 1956 1956
query60 358 323 311 311
query61 191 185 186 185
query62 447 355 308 308
query63 299 259 272 259
query64 4786 1310 994 994
query65 3797 3743 3687 3687
query66 1449 425 316 316
query67 15278 15342 14851 14851
query68 7501 988 719 719
query69 507 337 302 302
query70 1015 943 947 943
query71 381 309 269 269
query72 6216 4921 5222 4921
query73 742 663 307 307
query74 8785 8735 8666 8666
query75 3042 2916 2550 2550
query76 3949 1055 652 652
query77 517 371 267 267
query78 9685 9628 9100 9100
query79 1802 917 605 605
query80 683 575 483 483
query81 518 260 227 227
query82 246 140 113 113
query83 267 254 234 234
query84 265 125 106 106
query85 915 499 468 468
query86 375 324 323 323
query87 2888 2832 2759 2759
query88 4457 2244 2258 2244
query89 389 350 333 333
query90 2159 150 138 138
query91 171 161 144 144
query92 85 75 61 61
query93 2402 895 553 553
query94 579 311 287 287
query95 570 330 354 330
query96 587 473 202 202
query97 2293 2405 2306 2306
query98 232 202 193 193
query99 589 568 524 524
Total cold run time: 256193 ms
Total hot run time: 174093 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713828104
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b119cd5b30259d7c46ff99fdec66478a31180761_b119cd5b30259d7c46ff99fdec66478a31180761_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b119cd5b30259d7c46ff99fdec66478a31180761_b119cd5b30259d7c46ff99fdec66478a31180761_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.57% (1780/2237) |
| Line Coverage | 64.86% (31538/48624) |
| Region Coverage | 65.39% (15685/23986) |
| Branch Coverage | 55.99% (8334/14886) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717377943
<details>
<summary>TPC-DS: <b>Total hot run time: 173572 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 797a4d615363ccd28721ca73fb3a2cd51941185b, data reload: false
query5 4395 604 425 425
query6 323 221 208 208
query7 4209 452 259 259
query8 336 278 248 248
query9 8724 2608 2633 2608
query10 510 390 308 308
query11 15191 15230 15971 15230
query12 177 117 132 117
query13 1306 499 413 413
query14 6140 3088 2883 2883
query14_1 2704 2748 2870 2748
query15 205 200 177 177
query16 990 470 438 438
query17 1104 694 628 628
query18 3097 574 364 364
query19 234 234 203 203
query20 125 125 121 121
query21 216 142 129 129
query22 4623 4644 4522 4522
query23 16347 15669 15228 15228
query23_1 15358 15427 15423 15423
query24 7387 1516 1160 1160
query24_1 1180 1168 1184 1168
query25 559 502 368 368
query26 1243 254 142 142
query27 2747 428 276 276
query28 4490 2090 2120 2090
query29 743 531 430 430
query30 301 237 205 205
query31 841 629 541 541
query32 70 61 58 58
query33 525 323 314 314
query34 876 864 519 519
query35 725 788 699 699
query36 858 894 811 811
query37 126 89 73 73
query38 2675 2672 2643 2643
query39 783 755 734 734
query39_1 732 698 698 698
query40 210 124 109 109
query41 62 58 57 57
query42 97 94 95 94
query43 453 453 410 410
query44 1283 703 709 703
query45 184 180 174 174
query46 847 952 602 602
query47 1440 1451 1381 1381
query48 295 321 235 235
query49 595 433 340 340
query50 623 266 205 205
query51 3838 3763 3774 3763
query52 107 103 98 98
query53 290 329 265 265
query54 274 268 234 234
query55 72 73 65 65
query56 280 280 278 278
query57 1054 1013 912 912
query58 264 249 250 249
query59 2111 2023 2021 2021
query60 302 307 285 285
query61 164 162 160 160
query62 403 355 310 310
query63 302 258 263 258
query64 4891 1280 972 972
query65 3794 3735 3725 3725
query66 1398 398 302 302
query67 15143 15009 15644 15009
query68 8211 972 692 692
query69 482 343 304 304
query70 1027 931 928 928
query71 371 297 270 270
query72 6075 3391 3520 3391
query73 761 711 297 297
query74 8781 8792 8624 8624
query75 3126 2840 2429 2429
query76 4094 1062 641 641
query77 819 357 281 281
query78 9756 9981 9216 9216
query79 1247 903 561 561
query80 653 551 476 476
query81 515 269 238 238
query82 228 145 111 111
query83 256 246 240 240
query84 260 121 106 106
query85 900 516 465 465
query86 399 312 326 312
query87 2896 2877 2751 2751
query88 3123 2214 2204 2204
query89 389 346 313 313
query90 2194 148 142 142
query91 179 174 138 138
query92 78 63 63 63
query93 1557 905 530 530
query94 536 325 287 287
query95 558 373 306 306
query96 591 450 202 202
query97 2274 2357 2331 2331
query98 236 215 200 200
query99 572 570 483 483
Total cold run time: 256286 ms
Total hot run time: 173572 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3749429373
<details>
<summary>TPC-H: <b>Total hot run time: 31511 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f, data reload: false
------ Round 1 ----------------------------------
q1 17600 4642 4046 4046
q2 2030 361 236 236
q3 10163 1304 705 705
q4 10213 866 316 316
q5 7529 2099 1885 1885
q6 188 176 136 136
q7 945 790 664 664
q8 9293 1479 1202 1202
q9 4881 4617 4559 4559
q10 6814 1836 1440 1440
q11 523 286 271 271
q12 693 741 592 592
q13 17773 3832 3131 3131
q14 300 309 275 275
q15 600 511 504 504
q16 668 670 632 632
q17 660 799 480 480
q18 6826 6562 6318 6318
q19 1109 967 615 615
q20 395 371 248 248
q21 3028 2410 2275 2275
q22 1091 1013 981 981
Total cold run time: 103322 ms
Total hot run time: 31511 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4100 4045 4057 4045
q2 340 399 302 302
q3 2093 2581 2259 2259
q4 1324 1745 1337 1337
q5 4112 4035 4013 4013
q6 211 168 128 128
q7 1874 1813 1875 1813
q8 2654 2481 2467 2467
q9 7215 7202 7245 7202
q10 2765 2851 2289 2289
q11 583 484 468 468
q12 731 778 648 648
q13 3684 3981 3533 3533
q14 312 311 296 296
q15 564 510 530 510
q16 643 678 655 655
q17 1207 1410 1422 1410
q18 8138 7957 7935 7935
q19 919 886 856 856
q20 1994 2085 1903 1903
q21 4871 4567 4428 4428
q22 1157 1067 972 972
Total cold run time: 51491 ms
Total hot run time: 49469 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717387540
<details>
<summary>ClickBench: <b>Total hot run time: 28.01 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 797a4d615363ccd28721ca73fb3a2cd51941185b, data reload: false
query1 0.05 0.05 0.05
query2 0.15 0.07 0.06
query3 0.34 0.08 0.08
query4 1.60 0.10 0.10
query5 0.26 0.24 0.25
query6 1.14 0.64 0.66
query7 0.03 0.02 0.03
query8 0.08 0.05 0.06
query9 0.58 0.52 0.49
query10 0.55 0.54 0.54
query11 0.26 0.13 0.13
query12 0.25 0.14 0.13
query13 0.62 0.61 0.60
query14 0.98 0.97 0.97
query15 0.86 0.82 0.81
query16 0.40 0.39 0.39
query17 1.02 1.06 1.06
query18 0.24 0.22 0.21
query19 1.95 1.80 1.88
query20 0.02 0.02 0.02
query21 15.40 0.28 0.24
query22 5.01 0.10 0.10
query23 15.40 0.40 0.22
query24 2.39 0.46 0.31
query25 0.10 0.09 0.11
query26 0.19 0.18 0.18
query27 0.10 0.09 0.09
query28 3.67 1.13 0.97
query29 12.55 4.04 3.28
query30 0.33 0.13 0.11
query31 2.82 0.70 0.45
query32 3.25 0.62 0.50
query33 3.04 3.10 3.06
query34 16.89 5.09 4.48
query35 4.49 4.54 4.58
query36 0.62 0.49 0.48
query37 0.25 0.08 0.09
query38 0.20 0.05 0.06
query39 0.08 0.05 0.05
query40 0.20 0.18 0.17
query41 0.13 0.06 0.06
query42 0.08 0.05 0.05
query43 0.06 0.06 0.06
Total cold run time: 98.63 s
Total hot run time: 28.01 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3794179412
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3794155632
# BE Regression && UT Coverage Report
Increment line coverage `86.13% (354/411)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.41% (25463/35656) |
| Line Coverage | 53.96% (266488/493858) |
| Region Coverage | 51.62% (221151/428380) |
| Branch Coverage | 52.96% (94843/179070) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698212198
<details>
<summary>ClickBench: <b>Total hot run time: 27.92 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c82a283be5705daae1e30b7d0b1d8012e62fca31, data reload: false
query1 0.06 0.05 0.05
query2 0.13 0.06 0.07
query3 0.34 0.08 0.08
query4 1.61 0.09 0.10
query5 0.27 0.25 0.25
query6 1.14 0.66 0.66
query7 0.03 0.02 0.03
query8 0.09 0.06 0.06
query9 0.58 0.51 0.50
query10 0.55 0.55 0.54
query11 0.27 0.14 0.14
query12 0.26 0.15 0.15
query13 0.63 0.62 0.61
query14 1.02 1.02 1.02
query15 0.87 0.82 0.81
query16 0.40 0.40 0.39
query17 1.06 0.98 1.03
query18 0.23 0.22 0.21
query19 1.86 1.84 1.88
query20 0.02 0.01 0.02
query21 15.39 0.28 0.24
query22 4.98 0.10 0.09
query23 15.41 0.39 0.23
query24 2.40 0.47 0.30
query25 0.10 0.09 0.09
query26 0.18 0.18 0.17
query27 0.10 0.10 0.09
query28 3.82 1.12 0.98
query29 12.59 4.00 3.24
query30 0.42 0.13 0.11
query31 2.81 0.64 0.43
query32 3.24 0.60 0.48
query33 2.98 2.99 3.06
query34 16.96 5.12 4.52
query35 4.50 4.57 4.47
query36 0.61 0.50 0.49
query37 0.26 0.08 0.08
query38 0.21 0.06 0.07
query39 0.07 0.05 0.05
query40 0.21 0.18 0.17
query41 0.14 0.06 0.06
query42 0.08 0.05 0.05
query43 0.06 0.06 0.06
Total cold run time: 98.94 s
Total hot run time: 27.92 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3697107599
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a8f6d42fbc7af5425b4412fbf5ef84b6851e145b_a8f6d42fbc7af5425b4412fbf5ef84b6851e145b_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a8f6d42fbc7af5425b4412fbf5ef84b6851e145b_a8f6d42fbc7af5425b4412fbf5ef84b6851e145b_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.52% (1771/2227) |
| Line Coverage | 64.82% (31306/48299) |
| Region Coverage | 65.39% (15582/23831) |
| Branch Coverage | 55.98% (8283/14796) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3697313339
<details>
<summary>ClickBench: <b>Total hot run time: 26.98 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a8f6d42fbc7af5425b4412fbf5ef84b6851e145b, data reload: false
query1 0.05 0.04 0.05
query2 0.11 0.05 0.05
query3 0.26 0.08 0.09
query4 1.61 0.12 0.11
query5 0.29 0.28 0.26
query6 1.15 0.67 0.65
query7 0.03 0.02 0.02
query8 0.05 0.04 0.05
query9 0.57 0.51 0.51
query10 0.55 0.55 0.56
query11 0.16 0.11 0.11
query12 0.16 0.12 0.13
query13 0.61 0.59 0.61
query14 0.99 0.98 0.99
query15 0.82 0.79 0.80
query16 0.41 0.40 0.40
query17 1.07 1.06 1.07
query18 0.23 0.21 0.21
query19 1.97 1.85 1.86
query20 0.02 0.01 0.01
query21 15.44 0.31 0.13
query22 4.61 0.06 0.04
query23 16.03 0.29 0.10
query24 1.24 0.27 0.18
query25 0.09 0.05 0.06
query26 0.15 0.13 0.14
query27 0.08 0.06 0.05
query28 3.00 1.06 0.89
query29 12.64 3.85 3.14
query30 0.27 0.15 0.12
query31 2.82 0.61 0.37
query32 3.23 0.55 0.46
query33 2.97 3.14 3.04
query34 16.60 5.20 4.47
query35 4.49 4.46 4.95
query36 0.71 0.55 0.57
query37 0.12 0.07 0.07
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.16 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 96.05 s
Total hot run time: 26.98 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698133382
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3712156442
# FE Regression Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_d038e2ebb872beac2bf3196c7dca50460e9d3d9e_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_d038e2ebb872beac2bf3196c7dca50460e9d3d9e_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713013447
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/d130bbc50fa98b3b61631d6dd449cd7882bd60f7_d130bbc50fa98b3b61631d6dd449cd7882bd60f7_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/d130bbc50fa98b3b61631d6dd449cd7882bd60f7_d130bbc50fa98b3b61631d6dd449cd7882bd60f7_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.57% (1780/2237) |
| Line Coverage | 64.87% (31542/48624) |
| Region Coverage | 65.40% (15688/23986) |
| Branch Coverage | 56.05% (8343/14886) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3714406636
# BE Regression && UT Coverage Report
Increment line coverage `90.43% (378/418)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_b119cd5b30259d7c46ff99fdec66478a31180761_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_b119cd5b30259d7c46ff99fdec66478a31180761_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 66.02% (23021/34871) |
| Line Coverage | 52.82% (236450/447640) |
| Region Coverage | 48.18% (196592/408004) |
| Branch Coverage | 49.03% (83353/170019) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3749138831
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f_1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f_1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.57% (1784/2242) |
| Line Coverage | 64.79% (31744/48997) |
| Region Coverage | 65.44% (15784/24120) |
| Branch Coverage | 56.01% (8384/14968) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on code in PR #59307:
URL: https://github.com/apache/doris/pull/59307#discussion_r2707969170
##########
be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:
##########
@@ -305,6 +495,32 @@ void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_reserve_decompress_buf(siz
}
}
+template <bool IN_COLLECTION, bool OFFSET_INDEX>
+void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_insert_page_into_cache(
+ const std::vector<uint8_t>& level_bytes, const Slice& payload) {
+ StoragePageCache::CacheKey key(
+ fmt::format("{}::{}", _stream_reader->path(), _stream_reader->mtime()),
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717241227
# FE UT Coverage Report
Increment line coverage ` 100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_797a4d615363ccd28721ca73fb3a2cd51941185b/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3797686823
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752959570
<details>
<summary>TPC-H: <b>Total hot run time: 31504 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7cb3056f1e37e48a937fe3060608b5e62e84c8d5, data reload: false
------ Round 1 ----------------------------------
q1 17600 4254 4069 4069
q2 2074 356 238 238
q3 10398 1259 718 718
q4 10340 824 301 301
q5 9230 2042 1936 1936
q6 223 173 137 137
q7 915 807 670 670
q8 9274 1433 1177 1177
q9 5028 4650 4672 4650
q10 6831 1793 1410 1410
q11 503 273 283 273
q12 749 729 591 591
q13 17815 3845 3054 3054
q14 288 296 291 291
q15 580 515 503 503
q16 677 665 616 616
q17 668 850 464 464
q18 6627 6415 6337 6337
q19 1114 992 645 645
q20 401 360 241 241
q21 3021 2467 2231 2231
q22 1049 1028 952 952
Total cold run time: 105405 ms
Total hot run time: 31504 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4168 4046 4048 4046
q2 332 403 313 313
q3 2087 2601 2223 2223
q4 1303 1754 1308 1308
q5 4127 4001 4054 4001
q6 210 175 128 128
q7 1873 1805 2140 1805
q8 2658 2509 2371 2371
q9 7395 7235 7265 7235
q10 2578 2692 2344 2344
q11 560 465 468 465
q12 755 764 696 696
q13 3610 4229 3271 3271
q14 310 321 300 300
q15 556 507 497 497
q16 667 700 663 663
q17 1252 1359 1354 1354
q18 8012 7964 7778 7778
q19 849 844 826 826
q20 1986 2037 1894 1894
q21 4531 4209 4176 4176
q22 1083 1056 972 972
Total cold run time: 50902 ms
Total hot run time: 48666 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713756010
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3687764213
<details>
<summary>ClickBench: <b>Total hot run time: 27.34 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e32d9d4931d69e255e597e9d8c195bdeddb732d2, data reload: false
query1 0.05 0.05 0.05
query2 0.09 0.05 0.05
query3 0.25 0.08 0.08
query4 1.60 0.12 0.12
query5 0.27 0.25 0.24
query6 1.19 0.64 0.64
query7 0.03 0.03 0.02
query8 0.05 0.04 0.04
query9 0.55 0.50 0.49
query10 0.55 0.56 0.56
query11 0.15 0.11 0.12
query12 0.15 0.11 0.12
query13 0.62 0.60 0.60
query14 0.98 1.00 0.99
query15 0.83 0.81 0.81
query16 0.42 0.42 0.39
query17 0.99 1.06 1.02
query18 0.23 0.21 0.22
query19 1.87 1.85 1.84
query20 0.02 0.01 0.02
query21 15.43 0.28 0.13
query22 4.84 0.05 0.05
query23 16.01 0.29 0.10
query24 1.00 0.29 0.46
query25 0.09 0.05 0.11
query26 0.14 0.13 0.13
query27 0.09 0.08 0.07
query28 4.05 1.22 1.03
query29 12.64 4.03 3.21
query30 0.27 0.14 0.11
query31 2.81 0.62 0.39
query32 3.24 0.55 0.45
query33 2.98 3.02 3.06
query34 16.70 5.17 4.54
query35 4.57 4.54 4.55
query36 0.64 0.50 0.51
query37 0.11 0.07 0.07
query38 0.08 0.04 0.04
query39 0.04 0.02 0.02
query40 0.17 0.15 0.13
query41 0.09 0.04 0.03
query42 0.05 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 96.97 s
Total hot run time: 27.34 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3694798695
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3694887072
<details>
<summary>ClickBench: <b>Total hot run time: 27.13 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 59ebab3389bf5deaea3a89564cce010b55705628, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.04
query3 0.26 0.08 0.08
query4 1.60 0.11 0.11
query5 0.28 0.26 0.27
query6 1.16 0.64 0.64
query7 0.03 0.03 0.02
query8 0.06 0.05 0.05
query9 0.59 0.50 0.51
query10 0.55 0.55 0.55
query11 0.15 0.11 0.11
query12 0.15 0.11 0.12
query13 0.64 0.59 0.62
query14 1.00 0.97 0.98
query15 0.80 0.79 0.80
query16 0.39 0.39 0.41
query17 0.98 0.96 1.04
query18 0.23 0.22 0.21
query19 1.90 1.79 1.89
query20 0.01 0.01 0.02
query21 15.45 0.28 0.13
query22 4.96 0.05 0.05
query23 16.17 0.29 0.11
query24 1.19 0.34 0.41
query25 0.07 0.06 0.04
query26 0.14 0.13 0.13
query27 0.08 0.06 0.06
query28 4.09 1.22 1.04
query29 12.59 4.08 3.17
query30 0.30 0.13 0.15
query31 2.83 0.63 0.39
query32 3.23 0.55 0.45
query33 2.97 2.96 2.98
query34 16.61 5.12 4.54
query35 4.54 4.54 4.65
query36 0.66 0.49 0.49
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.04 0.02 0.03
query40 0.17 0.14 0.13
query41 0.09 0.04 0.03
query42 0.04 0.03 0.02
query43 0.04 0.03 0.03
Total cold run time: 97.38 s
Total hot run time: 27.13 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3748808064
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710721408
<details>
<summary>TPC-H: <b>Total hot run time: 31339 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d038e2ebb872beac2bf3196c7dca50460e9d3d9e, data reload: false
------ Round 1 ----------------------------------
q1 16461 4206 4027 4027
q2 2037 364 265 265
q3 9638 1219 719 719
q4 9783 865 320 320
q5 7400 2081 1885 1885
q6 186 167 143 143
q7 957 803 648 648
q8 9083 1366 1116 1116
q9 4736 4590 4560 4560
q10 6756 1807 1390 1390
q11 491 307 275 275
q12 669 753 586 586
q13 17590 3865 3060 3060
q14 289 290 272 272
q15 561 518 503 503
q16 697 684 636 636
q17 666 774 588 588
q18 6561 6395 6310 6310
q19 1111 970 602 602
q20 405 368 253 253
q21 2997 2506 2219 2219
q22 1037 992 962 962
Total cold run time: 100111 ms
Total hot run time: 31339 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4133 4033 4041 4033
q2 323 382 311 311
q3 2092 2606 2179 2179
q4 1338 1770 1272 1272
q5 4015 3961 3967 3961
q6 210 171 132 132
q7 1848 1824 1649 1649
q8 2425 2475 2508 2475
q9 7335 7133 7120 7120
q10 2407 2762 2329 2329
q11 546 517 466 466
q12 704 769 652 652
q13 3649 4084 3387 3387
q14 296 322 278 278
q15 556 511 516 511
q16 650 689 672 672
q17 1276 1379 1405 1379
q18 8136 7848 7681 7681
q19 863 867 877 867
q20 2068 2059 1907 1907
q21 4776 4610 4278 4278
q22 1091 996 977 977
Total cold run time: 50737 ms
Total hot run time: 48516 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3713031398
# FE UT Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_d130bbc50fa98b3b61631d6dd449cd7882bd60f7/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_d130bbc50fa98b3b61631d6dd449cd7882bd60f7/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689715675
<details>
<summary>TPC-H: <b>Total hot run time: 36141 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7a15d5f440c5328181e229a72f8fd4a7959ca301, data reload: false
------ Round 1 ----------------------------------
q1 17622 4409 4033 4033
q2 2025 352 240 240
q3 10186 1317 740 740
q4 10208 831 311 311
q5 7534 2213 1882 1882
q6 184 163 133 133
q7 1051 882 722 722
q8 9348 1451 1211 1211
q9 7103 5372 5361 5361
q10 6854 2410 1965 1965
q11 534 323 298 298
q12 658 741 559 559
q13 17799 3665 3023 3023
q14 302 286 271 271
q15 585 524 513 513
q16 669 685 636 636
q17 684 773 596 596
q18 7464 7464 7934 7464
q19 1189 1012 662 662
q20 441 378 257 257
q21 4542 4230 4235 4230
q22 1135 1097 1034 1034
Total cold run time: 108117 ms
Total hot run time: 36141 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4351 4261 4306 4261
q2 345 421 329 329
q3 2416 2924 2413 2413
q4 1646 1856 1384 1384
q5 4513 4619 4468 4468
q6 213 175 126 126
q7 2128 1929 1833 1833
q8 2732 2450 2458 2450
q9 7578 7433 7532 7433
q10 2915 3125 2618 2618
q11 604 512 490 490
q12 643 704 591 591
q13 3284 3651 3004 3004
q14 266 280 263 263
q15 538 488 485 485
q16 619 651 610 610
q17 1111 1430 1372 1372
q18 7375 7070 7121 7070
q19 835 809 835 809
q20 1938 1935 1784 1784
q21 4616 4271 4245 4245
q22 1132 1018 995 995
Total cold run time: 51798 ms
Total hot run time: 49033 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3698363246
# FE Regression Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c82a283be5705daae1e30b7d0b1d8012e62fca31_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c82a283be5705daae1e30b7d0b1d8012e62fca31_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3747925846
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3749469407
<details>
<summary>TPC-DS: <b>Total hot run time: 173409 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1dc221f3e4e20aec41a7bc85fdfcc0068b35a26f, data reload: false
query5 4927 594 478 478
query6 328 224 214 214
query7 4218 457 257 257
query8 343 260 233 233
query9 8708 2852 2865 2852
query10 556 407 333 333
query11 15135 15044 14874 14874
query12 178 118 117 117
query13 1268 486 371 371
query14 6480 2992 2714 2714
query14_1 2655 2763 2673 2673
query15 201 197 173 173
query16 1002 479 472 472
query17 1077 678 575 575
query18 2668 434 337 337
query19 231 221 195 195
query20 124 116 114 114
query21 222 138 118 118
query22 4031 4020 3834 3834
query23 16040 15729 15330 15330
query23_1 15749 15583 15484 15484
query24 7141 1522 1153 1153
query24_1 1196 1158 1156 1156
query25 571 472 419 419
query26 1245 267 161 161
query27 2745 444 282 282
query28 4535 2117 2113 2113
query29 824 552 448 448
query30 306 232 216 216
query31 774 610 559 559
query32 84 81 76 76
query33 544 362 321 321
query34 910 877 514 514
query35 720 755 721 721
query36 889 916 824 824
query37 130 91 80 80
query38 2690 2716 2632 2632
query39 759 749 732 732
query39_1 712 711 710 710
query40 224 136 119 119
query41 65 68 62 62
query42 102 99 101 99
query43 429 443 411 411
query44 1310 724 731 724
query45 184 183 172 172
query46 820 937 571 571
query47 1446 1415 1432 1415
query48 314 319 235 235
query49 616 418 335 335
query50 614 276 205 205
query51 3722 3850 3759 3759
query52 102 105 94 94
query53 289 317 269 269
query54 297 258 251 251
query55 85 78 76 76
query56 297 313 307 307
query57 980 1031 960 960
query58 274 261 269 261
query59 1943 2225 2158 2158
query60 329 332 316 316
query61 165 162 157 157
query62 396 354 315 315
query63 293 254 260 254
query64 4948 1285 987 987
query65 3777 3761 3737 3737
query66 1394 417 308 308
query67 15676 14851 15129 14851
query68 7011 973 723 723
query69 522 365 322 322
query70 1060 966 885 885
query71 352 320 278 278
query72 5676 3363 3402 3363
query73 785 725 302 302
query74 8855 8686 8707 8686
query75 2809 2770 2423 2423
query76 3411 1053 634 634
query77 564 368 302 302
query78 9782 9830 9106 9106
query79 1636 885 569 569
query80 668 568 488 488
query81 513 262 228 228
query82 211 143 110 110
query83 256 260 240 240
query84 256 109 104 104
query85 887 513 461 461
query86 348 294 262 262
query87 2847 2823 2731 2731
query88 3416 2556 2545 2545
query89 383 350 318 318
query90 1985 171 163 163
query91 175 164 138 138
query92 83 70 69 69
query93 1048 911 519 519
query94 564 307 309 307
query95 583 389 313 313
query96 637 498 223 223
query97 2350 2439 2291 2291
query98 215 196 196 196
query99 598 553 510 510
Total cold run time: 253787 ms
Total hot run time: 173409 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3753110380
# BE UT Coverage Report
Increment line coverage `81.67% (343/420)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/7cb3056f1e37e48a937fe3060608b5e62e84c8d5_7cb3056f1e37e48a937fe3060608b5e62e84c8d5/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/7cb3056f1e37e48a937fe3060608b5e62e84c8d5_7cb3056f1e37e48a937fe3060608b5e62e84c8d5/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.00% (19011/35871) |
| Line Coverage | 39.05% (176186/451131) |
| Region Coverage | 33.62% (136362/405639) |
| Branch Coverage | 34.66% (58954/170105) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3697030290
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717194161
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/797a4d615363ccd28721ca73fb3a2cd51941185b_797a4d615363ccd28721ca73fb3a2cd51941185b_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/797a4d615363ccd28721ca73fb3a2cd51941185b_797a4d615363ccd28721ca73fb3a2cd51941185b_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.53% (1779/2237) |
| Line Coverage | 64.72% (31481/48643) |
| Region Coverage | 65.38% (15654/23943) |
| Branch Coverage | 55.97% (8315/14856) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3687397239
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3687701804
<details>
<summary>TPC-H: <b>Total hot run time: 35539 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e32d9d4931d69e255e597e9d8c195bdeddb732d2, data reload: false
------ Round 1 ----------------------------------
q1 17659 4414 4144 4144
q2 2035 362 237 237
q3 10167 1359 792 792
q4 10208 839 319 319
q5 7608 2213 1914 1914
q6 203 172 137 137
q7 1019 897 724 724
q8 9367 1479 1143 1143
q9 7249 5332 5313 5313
q10 6891 2412 2000 2000
q11 527 341 305 305
q12 726 747 564 564
q13 17801 3734 3105 3105
q14 297 294 277 277
q15 606 515 504 504
q16 712 699 642 642
q17 686 793 586 586
q18 7632 7129 7120 7120
q19 1113 982 650 650
q20 399 367 249 249
q21 4294 3958 3866 3866
q22 1021 1017 948 948
Total cold run time: 108220 ms
Total hot run time: 35539 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4133 4045 4074 4045
q2 345 421 323 323
q3 2185 2727 2283 2283
q4 1380 1780 1327 1327
q5 4311 4776 4764 4764
q6 222 174 138 138
q7 2219 1998 1903 1903
q8 2813 2570 2540 2540
q9 7594 7466 7458 7458
q10 3128 3292 2848 2848
q11 614 547 535 535
q12 715 736 803 736
q13 3652 3983 3445 3445
q14 305 317 278 278
q15 567 525 510 510
q16 686 707 630 630
q17 1252 1481 1583 1481
q18 7888 7810 7484 7484
q19 918 922 927 922
q20 2045 1964 1812 1812
q21 4696 4306 4257 4257
q22 1053 1007 996 996
Total cold run time: 52721 ms
Total hot run time: 50715 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689083161
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689167280
<details>
<summary>TPC-DS: <b>Total hot run time: 179518 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b25ce38cc5917e7ea076b672f14333bd0b196779, data reload: false
query5 5033 601 451 451
query6 339 226 218 218
query7 4225 456 273 273
query8 320 253 228 228
query9 8813 2529 2523 2523
query10 532 398 328 328
query11 15452 15016 14734 14734
query12 180 119 115 115
query13 1253 497 390 390
query14 5897 2986 2767 2767
query14_1 2689 2611 2653 2611
query15 208 193 177 177
query16 819 472 450 450
query17 1088 681 589 589
query18 2452 431 337 337
query19 235 222 201 201
query20 125 120 112 112
query21 229 141 118 118
query22 3949 4068 3939 3939
query23 16746 16298 16144 16144
query23_1 16097 16162 16103 16103
query24 7344 1660 1244 1244
query24_1 1222 1210 1223 1210
query25 542 479 426 426
query26 1258 267 168 168
query27 2754 479 301 301
query28 4456 2137 2155 2137
query29 843 570 482 482
query30 325 246 215 215
query31 804 698 614 614
query32 84 72 68 68
query33 552 343 301 301
query34 925 899 545 545
query35 788 806 743 743
query36 858 899 848 848
query37 128 94 81 81
query38 3031 2998 2967 2967
query39 789 771 911 771
query39_1 698 720 693 693
query40 227 141 119 119
query41 67 62 61 61
query42 111 102 104 102
query43 432 441 428 428
query44 1308 757 731 731
query45 192 193 182 182
query46 886 982 609 609
query47 1665 1708 1635 1635
query48 317 322 242 242
query49 620 425 348 348
query50 667 302 217 217
query51 3849 3860 3788 3788
query52 101 106 98 98
query53 319 350 299 299
query54 284 265 246 246
query55 82 80 72 72
query56 292 289 310 289
query57 1195 1135 1117 1117
query58 288 260 249 249
query59 2412 2445 2397 2397
query60 352 314 293 293
query61 166 193 165 165
query62 770 748 678 678
query63 329 300 315 300
query64 5010 1319 1008 1008
query65 4058 3967 3971 3967
query66 1444 433 310 310
query67 15563 14999 14913 14913
query68 2814 1018 755 755
query69 440 351 310 310
query70 1064 985 975 975
query71 324 300 274 274
query72 6089 5143 5339 5143
query73 532 623 318 318
query74 8853 9012 8707 8707
query75 3168 3173 2810 2810
query76 2878 1166 727 727
query77 381 402 297 297
query78 9436 9619 8826 8826
query79 1394 897 613 613
query80 1624 663 560 560
query81 555 266 230 230
query82 643 133 99 99
query83 378 257 238 238
query84 266 117 101 101
query85 957 513 471 471
query86 388 307 287 287
query87 3277 3190 3083 3083
query88 3281 2273 2272 2272
query89 477 433 400 400
query90 1994 156 162 156
query91 174 165 146 146
query92 74 68 62 62
query93 1088 909 559 559
query94 576 311 254 254
query95 560 325 358 325
query96 581 458 210 210
query97 2260 2297 2199 2199
query98 213 201 197 197
query99 1356 1342 1286 1286
Total cold run time: 255635 ms
Total hot run time: 179518 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689744067
<details>
<summary>ClickBench: <b>Total hot run time: 27.16 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7a15d5f440c5328181e229a72f8fd4a7959ca301, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.05 0.05
query3 0.26 0.09 0.08
query4 1.61 0.11 0.12
query5 0.27 0.26 0.26
query6 1.16 0.64 0.63
query7 0.03 0.02 0.03
query8 0.05 0.04 0.05
query9 0.56 0.51 0.51
query10 0.55 0.55 0.54
query11 0.16 0.11 0.10
query12 0.14 0.11 0.12
query13 0.61 0.60 0.60
query14 0.99 0.99 0.98
query15 0.82 0.79 0.81
query16 0.40 0.41 0.41
query17 0.96 1.02 1.06
query18 0.22 0.22 0.21
query19 1.81 1.79 1.79
query20 0.01 0.01 0.01
query21 15.42 0.26 0.13
query22 4.98 0.05 0.04
query23 15.91 0.29 0.10
query24 0.94 0.22 0.54
query25 0.09 0.06 0.04
query26 0.15 0.14 0.13
query27 0.08 0.06 0.05
query28 3.90 1.21 1.02
query29 12.57 3.93 3.23
query30 0.28 0.14 0.12
query31 2.82 0.63 0.39
query32 3.22 0.56 0.46
query33 2.96 3.02 2.96
query34 16.97 5.18 4.56
query35 4.60 4.56 4.56
query36 0.66 0.49 0.49
query37 0.10 0.07 0.06
query38 0.07 0.03 0.04
query39 0.04 0.03 0.03
query40 0.16 0.14 0.14
query41 0.09 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 96.85 s
Total hot run time: 27.16 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3692809571
# BE Regression && UT Coverage Report
Increment line coverage `96.95% (254/262)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_3d89ddaf5ceb906c64be3fa8b00e443b9fbf7e45_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.86% (25634/34706) |
| Line Coverage | 61.27% (273709/446732) |
| Region Coverage | 56.16% (228172/406316) |
| Branch Coverage | 58.02% (98391/169593) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710147866
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3712144171
# BE Regression && UT Coverage Report
Increment line coverage `88.78% (356/401)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_d038e2ebb872beac2bf3196c7dca50460e9d3d9e_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_d038e2ebb872beac2bf3196c7dca50460e9d3d9e_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.19% (25171/34867) |
| Line Coverage | 58.99% (264218/447922) |
| Region Coverage | 53.78% (219425/407979) |
| Branch Coverage | 55.35% (94088/170002) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3772994696
<details>
<summary>TPC-H: <b>Total hot run time: 32151 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c18b9a055f056a4348a0abe92bb480bbfa927357, data reload: false
------ Round 1 ----------------------------------
q1 17682 4239 4052 4052
q2 2022 351 252 252
q3 10164 1267 717 717
q4 10207 800 306 306
q5 7614 1993 1967 1967
q6 236 168 134 134
q7 922 781 648 648
q8 9261 1380 1147 1147
q9 5107 4707 4620 4620
q10 6827 1816 1389 1389
q11 503 306 284 284
q12 762 753 603 603
q13 17805 3848 3108 3108
q14 289 287 275 275
q15 605 516 499 499
q16 689 686 659 659
q17 989 798 500 500
q18 6715 6582 7148 6582
q19 1506 1039 690 690
q20 414 384 263 263
q21 3258 2644 2520 2520
q22 1136 1098 936 936
Total cold run time: 104713 ms
Total hot run time: 32151 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4485 4262 4442 4262
q2 326 403 308 308
q3 2310 2793 2378 2378
q4 1499 1920 1530 1530
q5 4403 4410 4329 4329
q6 213 164 131 131
q7 2025 1870 1759 1759
q8 2632 2383 2366 2366
q9 7306 7241 7090 7090
q10 2557 2706 2240 2240
q11 556 456 448 448
q12 682 736 645 645
q13 3753 3934 3127 3127
q14 261 281 265 265
q15 528 479 482 479
q16 627 653 598 598
q17 1111 1270 1351 1270
q18 7521 7342 7218 7218
q19 851 777 783 777
q20 1924 1964 1799 1799
q21 4455 4329 4161 4161
q22 1068 1001 967 967
Total cold run time: 51093 ms
Total hot run time: 48147 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752991872
<details>
<summary>ClickBench: <b>Total hot run time: 27.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7cb3056f1e37e48a937fe3060608b5e62e84c8d5, data reload: false
query1 0.06 0.05 0.05
query2 0.14 0.06 0.06
query3 0.38 0.08 0.08
query4 1.61 0.10 0.10
query5 0.25 0.24 0.25
query6 1.14 0.66 0.66
query7 0.03 0.02 0.02
query8 0.07 0.06 0.06
query9 0.59 0.51 0.49
query10 0.54 0.55 0.55
query11 0.26 0.13 0.13
query12 0.26 0.14 0.13
query13 0.62 0.61 0.59
query14 0.99 0.99 0.97
query15 0.91 0.82 0.83
query16 0.38 0.40 0.39
query17 1.06 1.04 1.07
query18 0.24 0.23 0.23
query19 1.82 1.81 1.80
query20 0.03 0.01 0.02
query21 15.40 0.30 0.24
query22 4.97 0.10 0.09
query23 15.41 0.39 0.22
query24 2.63 0.51 0.35
query25 0.10 0.10 0.09
query26 0.18 0.18 0.16
query27 0.10 0.09 0.09
query28 3.76 1.13 0.96
query29 12.53 4.00 3.30
query30 0.32 0.12 0.10
query31 2.81 0.66 0.43
query32 3.23 0.63 0.50
query33 3.01 2.98 3.04
query34 16.27 5.07 4.51
query35 4.51 4.46 4.54
query36 0.61 0.49 0.50
query37 0.25 0.09 0.09
query38 0.20 0.06 0.06
query39 0.07 0.05 0.05
query40 0.20 0.17 0.16
query41 0.13 0.06 0.06
query42 0.08 0.05 0.05
query43 0.06 0.06 0.05
Total cold run time: 98.21 s
Total hot run time: 27.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3752483507
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3773895369
# BE Regression && UT Coverage Report
Increment line coverage `86.13% (354/411)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59307_c18b9a055f056a4348a0abe92bb480bbfa927357_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.41% (25463/35656) |
| Line Coverage | 53.96% (266488/493858) |
| Region Coverage | 51.62% (221151/428380) |
| Branch Coverage | 52.96% (94843/179070) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #59307:
URL: https://github.com/apache/doris/pull/59307#discussion_r2706700447
##########
be/src/vec/exec/format/parquet/vparquet_page_reader.cpp:
##########
@@ -77,11 +83,79 @@ Status PageReader<IN_COLLECTION, OFFSET_INDEX>::parse_page_header() {
return Status::IOError("Should skip or load current page to get next page");
}
+ _page_statistics.page_read_counter += 1;
+
+ // Parse page header from file; header bytes are saved for possible cache insertion
const uint8_t* page_header_buf = nullptr;
size_t max_size = _end_offset - _offset;
size_t header_size = std::min(INIT_PAGE_HEADER_SIZE, max_size);
const size_t MAX_PAGE_HEADER_SIZE = config::parquet_header_max_size_mb << 20;
uint32_t real_header_size = 0;
+
+ // Try a header-only lookup in the page cache. Cached pages store
+ // header + optional v2 levels + uncompressed payload, so we can
+ // parse the page header directly from the cached bytes and avoid
+ // a file read for the header.
+ if (_page_read_ctx.enable_parquet_file_page_cache && !config::disable_storage_page_cache &&
+ StoragePageCache::instance() != nullptr) {
+ PageCacheHandle handle;
+ StoragePageCache::CacheKey key(fmt::format("{}::{}", _reader->path(), _reader->mtime()),
+ _end_offset, _offset);
+ if (StoragePageCache::instance()->lookup(key, &handle, segment_v2::DATA_PAGE)) {
+ // Parse header directly from cached data
+ _page_cache_handle = std::move(handle);
+ Slice s = _page_cache_handle.data();
+ real_header_size = cast_set<uint32_t>(s.size);
+ SCOPED_RAW_TIMER(&_page_statistics.decode_header_time);
+ auto st = deserialize_thrift_msg(reinterpret_cast<const uint8_t*>(s.data),
+ &real_header_size, true, &_cur_page_header);
+ if (!st.ok()) return st;
+ // Increment page cache counters for a true cache hit on header+payload
+ _page_statistics.page_cache_hit_counter += 1;
+ // Detect whether the cached payload is compressed or decompressed and record
+ bool is_cache_payload_decompressed = true;
+ if (_cur_page_header.compressed_page_size > 0) {
Review Comment:
Better extract this check logic:
```
bool should_cache_decompressed(const tparquet::PageHeader* header,
const tparquet::ColumnMetaData& metadata) {
if (header->compressed_page_size <= 0) return true;
if (metadata.codec == tparquet::CompressionCodec::UNCOMPRESSED) return true;
double ratio = static_cast<double>(header->uncompressed_page_size) /
static_cast<double>(header->compressed_page_size);
return ratio <= config::parquet_page_cache_decompress_threshold;
}
```
And reuse it for both here and in `be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp`
##########
be/src/io/file_factory.cpp:
##########
@@ -203,6 +203,21 @@ Result<io::FileReaderSPtr> FileFactory::create_file_reader(
const io::FileSystemProperties& system_properties,
const io::FileDescription& file_description, const io::FileReaderOptions& reader_options,
RuntimeProfile* profile) {
+ auto reader_res = _create_file_reader_internal(system_properties, file_description,
+ reader_options, profile);
+ if (!reader_res.has_value()) {
+ return unexpected(std::move(reader_res).error());
+ }
+ auto file_reader = std::move(reader_res).value();
+ LOG_INFO("create file reader for path={}, size={}, mtime={}", file_description.path,
Review Comment:
Remove this log, or using DEBUG level
##########
be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:
##########
@@ -305,6 +495,32 @@ void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_reserve_decompress_buf(siz
}
}
+template <bool IN_COLLECTION, bool OFFSET_INDEX>
+void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_insert_page_into_cache(
+ const std::vector<uint8_t>& level_bytes, const Slice& payload) {
+ StoragePageCache::CacheKey key(
+ fmt::format("{}::{}", _stream_reader->path(), _stream_reader->mtime()),
+ _page_reader->file_end_offset(), _page_reader->header_start_offset());
+ const std::vector<uint8_t>& header_bytes = _page_reader->header_bytes();
+ size_t total = header_bytes.size() + level_bytes.size() + payload.size;
+ auto* page = new DataPage(total, true, segment_v2::DATA_PAGE);
Review Comment:
Potential memory leak?
Better use a more safety way?
##########
be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:
##########
@@ -305,6 +495,32 @@ void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_reserve_decompress_buf(siz
}
}
+template <bool IN_COLLECTION, bool OFFSET_INDEX>
+void ColumnChunkReader<IN_COLLECTION, OFFSET_INDEX>::_insert_page_into_cache(
+ const std::vector<uint8_t>& level_bytes, const Slice& payload) {
+ StoragePageCache::CacheKey key(
+ fmt::format("{}::{}", _stream_reader->path(), _stream_reader->mtime()),
Review Comment:
`fmt::format("{}::{}", _stream_reader->path(), _stream_reader->mtime()`
this part is same for every page, better cache it to reuse:
```
class ParquetPageCacheKeyBuilder {
std::string _file_key_prefix; // Cached once per column chunk
public:
void init(const std::string& path, int64_t mtime) {
_file_key_prefix = fmt::format("{}::{}", path, mtime);
}
StoragePageCache::CacheKey make_key(uint64_t end_offset, uint64_t offset) const {
return StoragePageCache::CacheKey(_file_key_prefix, end_offset, offset);
}
};
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3772447527
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3797686913
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689141217
<details>
<summary>TPC-H: <b>Total hot run time: 35717 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b25ce38cc5917e7ea076b672f14333bd0b196779, data reload: false
------ Round 1 ----------------------------------
q1 17614 4146 4034 4034
q2 2032 359 239 239
q3 10157 1329 744 744
q4 10209 816 303 303
q5 7519 2127 1877 1877
q6 201 165 136 136
q7 1022 846 712 712
q8 9377 1374 1130 1130
q9 7022 5320 5285 5285
q10 6781 2381 1971 1971
q11 544 341 312 312
q12 678 751 572 572
q13 17792 3651 3032 3032
q14 292 299 273 273
q15 569 521 509 509
q16 711 677 617 617
q17 673 827 563 563
q18 7671 7360 8021 7360
q19 1120 978 622 622
q20 454 402 279 279
q21 4594 4448 4108 4108
q22 1129 1075 1039 1039
Total cold run time: 108161 ms
Total hot run time: 35717 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4250 4201 4169 4169
q2 340 433 342 342
q3 2337 2840 2421 2421
q4 1478 1842 1417 1417
q5 4564 4457 4491 4457
q6 215 170 129 129
q7 2039 2156 1788 1788
q8 2710 2522 2520 2520
q9 7529 7646 7497 7497
q10 3055 3274 2889 2889
q11 610 526 584 526
q12 688 740 608 608
q13 3295 3625 2976 2976
q14 264 283 257 257
q15 532 490 483 483
q16 662 653 606 606
q17 1103 1295 1347 1295
q18 7265 7101 7122 7101
q19 810 789 826 789
q20 1879 1957 1809 1809
q21 4595 4270 4202 4202
q22 1092 1036 961 961
Total cold run time: 51312 ms
Total hot run time: 49242 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3710483431
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3692502573
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3689599229
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717111453
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3717358405
<details>
<summary>TPC-H: <b>Total hot run time: 31349 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 797a4d615363ccd28721ca73fb3a2cd51941185b, data reload: false
------ Round 1 ----------------------------------
q1 17615 4217 4051 4051
q2 2013 362 244 244
q3 10178 1263 731 731
q4 10272 844 326 326
q5 8741 2138 1868 1868
q6 231 178 146 146
q7 914 800 666 666
q8 9287 1415 1105 1105
q9 5093 4767 4560 4560
q10 6838 1817 1406 1406
q11 520 294 270 270
q12 742 747 597 597
q13 17799 3814 3068 3068
q14 290 301 270 270
q15 601 512 502 502
q16 661 667 634 634
q17 659 784 530 530
q18 7259 6405 6226 6226
q19 1097 984 586 586
q20 399 360 247 247
q21 2972 2388 2345 2345
q22 1025 988 971 971
Total cold run time: 105206 ms
Total hot run time: 31349 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4120 4045 4068 4045
q2 328 410 319 319
q3 2116 2561 2270 2270
q4 1325 1763 1302 1302
q5 4136 3998 4068 3998
q6 211 169 133 133
q7 1880 1799 2036 1799
q8 2618 2544 2498 2498
q9 7171 7087 7199 7087
q10 2462 2718 2352 2352
q11 566 491 463 463
q12 702 751 652 652
q13 3686 4077 3351 3351
q14 364 339 280 280
q15 538 502 498 498
q16 641 682 645 645
q17 1346 1311 1342 1311
q18 7877 8003 7879 7879
q19 921 858 912 858
q20 1981 2096 1996 1996
q21 4780 4545 4633 4545
q22 1131 1056 1027 1027
Total cold run time: 50900 ms
Total hot run time: 49308 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](vparquet-reader) Implements parquet file page cache. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59307:
URL: https://github.com/apache/doris/pull/59307#issuecomment-3694884082
<details>
<summary>TPC-DS: <b>Total hot run time: 179267 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 59ebab3389bf5deaea3a89564cce010b55705628, data reload: false
query5 4924 618 458 458
query6 336 241 208 208
query7 4229 477 267 267
query8 315 256 244 244
query9 8781 2550 2542 2542
query10 540 367 329 329
query11 15542 14988 14736 14736
query12 177 118 118 118
query13 1275 511 394 394
query14 6107 3168 2815 2815
query14_1 2703 2735 2737 2735
query15 209 195 176 176
query16 1011 476 451 451
query17 1136 724 615 615
query18 2494 447 355 355
query19 233 234 213 213
query20 126 118 119 118
query21 224 144 118 118
query22 3864 4006 3808 3808
query23 16740 16255 16143 16143
query23_1 16103 16098 15979 15979
query24 7334 1657 1265 1265
query24_1 1260 1195 1268 1195
query25 569 497 446 446
query26 830 274 175 175
query27 2713 485 309 309
query28 4410 2130 2148 2130
query29 847 576 472 472
query30 305 244 220 220
query31 825 738 617 617
query32 86 71 76 71
query33 554 344 296 296
query34 885 896 558 558
query35 749 845 726 726
query36 901 909 819 819
query37 149 94 80 80
query38 3052 3096 2992 2992
query39 746 778 743 743
query39_1 705 704 704 704
query40 231 139 119 119
query41 67 61 62 61
query42 109 107 106 106
query43 438 445 402 402
query44 1363 754 746 746
query45 195 192 189 189
query46 870 981 624 624
query47 1642 1696 1624 1624
query48 320 335 254 254
query49 609 432 351 351
query50 693 327 227 227
query51 3833 3930 3824 3824
query52 109 113 99 99
query53 322 353 297 297
query54 287 268 248 248
query55 81 82 77 77
query56 289 295 295 295
query57 1141 1146 1096 1096
query58 269 252 252 252
query59 2362 2484 2331 2331
query60 324 315 293 293
query61 163 160 164 160
query62 743 709 661 661
query63 324 304 312 304
query64 4279 1305 996 996
query65 4021 3951 3933 3933
query66 1430 442 319 319
query67 15267 15132 14888 14888
query68 8358 988 714 714
query69 518 343 306 306
query70 1121 1032 998 998
query71 372 300 279 279
query72 6148 5001 5093 5001
query73 688 623 310 310
query74 8887 8964 8603 8603
query75 3213 3203 2823 2823
query76 3858 1128 744 744
query77 528 396 295 295
query78 9421 9723 8848 8848
query79 1270 859 613 613
query80 710 671 573 573
query81 544 271 234 234
query82 225 128 109 109
query83 266 259 242 242
query84 262 126 105 105
query85 913 519 474 474
query86 388 306 273 273
query87 3171 3162 3097 3097
query88 3300 2329 2317 2317
query89 479 418 406 406
query90 2169 162 153 153
query91 176 173 147 147
query92 87 68 65 65
query93 1648 949 554 554
query94 481 303 287 287
query95 593 382 305 305
query96 611 475 219 219
query97 2268 2292 2171 2171
query98 211 198 196 196
query99 1331 1347 1292 1292
Total cold run time: 260645 ms
Total hot run time: 179267 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org