You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "csun5285 (via GitHub)" <gi...@apache.org> on 2024/07/08 11:17:40 UTC
[PR] [opt] (inverted index) add inverted index file size for open file [doris]
csun5285 opened a new pull request, #37482:
URL: https://github.com/apache/doris/pull/37482
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2285279388
<details>
<summary>TPC-H: <b>Total hot run time: 39514 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false
------ Round 1 ----------------------------------
q1 18121 5000 4270 4270
q2 2007 204 203 203
q3 11300 993 1154 993
q4 10496 764 686 686
q5 7502 2573 2513 2513
q6 224 140 139 139
q7 969 605 611 605
q8 9228 1938 1955 1938
q9 8740 6548 6602 6548
q10 7044 2291 2198 2198
q11 459 243 248 243
q12 396 246 228 228
q13 17989 3005 2981 2981
q14 292 249 236 236
q15 516 488 489 488
q16 502 400 385 385
q17 992 684 722 684
q18 8154 7464 7498 7464
q19 6864 1088 919 919
q20 700 353 344 344
q21 5468 4445 4593 4445
q22 1108 1018 1004 1004
Total cold run time: 119071 ms
Total hot run time: 39514 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4450 4249 4238 4238
q2 384 277 279 277
q3 2904 2833 2829 2829
q4 1999 1868 1810 1810
q5 5476 5507 5530 5507
q6 235 140 137 137
q7 2199 1798 1800 1798
q8 3322 3463 3449 3449
q9 8839 8945 8940 8940
q10 3431 3177 3290 3177
q11 615 516 501 501
q12 850 640 651 640
q13 17423 3175 3174 3174
q14 338 291 295 291
q15 541 499 481 481
q16 511 421 433 421
q17 1840 1556 1539 1539
q18 8189 8135 7930 7930
q19 1762 1651 1581 1581
q20 2128 1925 1855 1855
q21 5561 5118 5394 5118
q22 1147 1061 1020 1020
Total cold run time: 74144 ms
Total hot run time: 56713 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2260264317
<details>
<summary>ClickBench: <b>Total hot run time: 30.39 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e34d3f2f2e8fc40588dd96971b4ffb5d1411f77, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.07 0.07
query5 0.48 0.47 0.49
query6 1.13 0.73 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.05
query9 0.58 0.52 0.54
query10 0.57 0.56 0.58
query11 0.15 0.12 0.11
query12 0.15 0.12 0.12
query13 0.62 0.61 0.60
query14 0.78 0.80 0.81
query15 0.91 0.88 0.86
query16 0.35 0.36 0.37
query17 1.01 1.02 1.00
query18 0.22 0.22 0.21
query19 1.83 1.81 1.75
query20 0.02 0.01 0.00
query21 15.40 0.78 0.65
query22 3.85 7.67 1.43
query23 18.08 1.31 1.30
query24 2.20 0.23 0.22
query25 0.18 0.08 0.08
query26 0.31 0.21 0.21
query27 0.46 0.24 0.22
query28 13.21 1.01 0.97
query29 12.56 3.37 3.38
query30 0.25 0.06 0.05
query31 2.87 0.41 0.41
query32 3.24 0.49 0.49
query33 2.99 3.00 3.01
query34 15.44 4.26 4.26
query35 4.29 4.30 4.30
query36 0.67 0.48 0.51
query37 0.18 0.16 0.16
query38 0.16 0.16 0.15
query39 0.04 0.04 0.04
query40 0.16 0.14 0.13
query41 0.11 0.06 0.06
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 107.63 s
Total hot run time: 30.39 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2213741277
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2217456941
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2262169007
<details>
<summary>ClickBench: <b>Total hot run time: 30.08 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 639d0bea360b4cb4807b38b9d5191d5cc5818713, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.69 0.07 0.06
query5 0.49 0.48 0.48
query6 1.15 0.72 0.71
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.56 0.50 0.51
query10 0.57 0.56 0.56
query11 0.15 0.12 0.11
query12 0.15 0.13 0.13
query13 0.62 0.60 0.60
query14 0.77 0.80 0.80
query15 0.91 0.86 0.86
query16 0.36 0.36 0.35
query17 0.99 1.03 1.01
query18 0.22 0.24 0.22
query19 1.83 1.74 1.74
query20 0.01 0.01 0.02
query21 15.39 0.80 0.66
query22 4.03 7.39 1.27
query23 17.88 1.31 1.30
query24 2.27 0.23 0.21
query25 0.18 0.09 0.08
query26 0.31 0.23 0.22
query27 0.45 0.24 0.23
query28 13.20 1.01 0.97
query29 12.51 3.28 3.26
query30 0.25 0.05 0.06
query31 2.88 0.41 0.41
query32 3.23 0.49 0.49
query33 2.99 2.96 3.02
query34 15.44 4.27 4.26
query35 4.26 4.27 4.29
query36 0.67 0.48 0.48
query37 0.19 0.17 0.16
query38 0.16 0.15 0.15
query39 0.05 0.03 0.03
query40 0.17 0.14 0.14
query41 0.10 0.05 0.05
query42 0.06 0.05 0.05
query43 0.05 0.05 0.05
Total cold run time: 107.6 s
Total hot run time: 30.08 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2270457234
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2262146426
<details>
<summary>TPC-H: <b>Total hot run time: 41959 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 639d0bea360b4cb4807b38b9d5191d5cc5818713, data reload: false
------ Round 1 ----------------------------------
q1 18410 4276 4225 4225
q2 2167 209 208 208
q3 13378 1275 1341 1275
q4 10732 855 940 855
q5 7641 3063 2989 2989
q6 224 136 137 136
q7 1044 616 614 614
q8 9454 1844 1991 1844
q9 8544 6613 6619 6613
q10 8742 3847 3853 3847
q11 429 263 259 259
q12 420 229 226 226
q13 17761 2962 2977 2962
q14 274 238 245 238
q15 532 477 477 477
q16 484 387 390 387
q17 986 954 937 937
q18 8099 7333 7279 7279
q19 1430 1221 1240 1221
q20 577 322 347 322
q21 5398 4769 4866 4769
q22 345 284 276 276
Total cold run time: 117071 ms
Total hot run time: 41959 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4103 4042 4064 4042
q2 329 227 218 218
q3 3009 3001 2995 2995
q4 1897 1918 1891 1891
q5 5304 5289 5235 5235
q6 217 130 136 130
q7 2050 1686 1745 1686
q8 3276 3376 3313 3313
q9 8371 8384 8357 8357
q10 3753 3831 3858 3831
q11 546 456 447 447
q12 739 577 527 527
q13 14962 2945 2987 2945
q14 284 255 260 255
q15 520 485 476 476
q16 461 390 412 390
q17 1765 1682 1687 1682
q18 7805 7285 7249 7249
q19 1704 1688 1695 1688
q20 1976 1766 1755 1755
q21 5461 5265 5216 5216
q22 532 458 475 458
Total cold run time: 69064 ms
Total hot run time: 54786 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on code in PR #37482:
URL: https://github.com/apache/doris/pull/37482#discussion_r1714991834
##########
be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:
##########
@@ -139,37 +127,44 @@ DorisCompoundReader::DorisCompoundReader(lucene::store::Directory* d, const char
entry->file_name = aid;
entry->offset = stream->readLong();
entry->length = stream->readLong();
- entries->put(aid, entry);
+ _entries->put(aid, entry);
// read header file data
if (entry->offset < 0) {
copyFile(entry->file_name.c_str(), entry->length, buffer, BUFFER_LENGTH);
}
Review Comment:
need to set debug point to throw error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2285292515
<details>
<summary>ClickBench: <b>Total hot run time: 31.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.22 0.06 0.05
query4 1.66 0.09 0.08
query5 0.50 0.51 0.49
query6 1.12 0.74 0.73
query7 0.02 0.01 0.01
query8 0.05 0.05 0.04
query9 0.55 0.49 0.49
query10 0.55 0.54 0.54
query11 0.15 0.11 0.11
query12 0.14 0.12 0.12
query13 0.63 0.59 0.59
query14 0.77 0.77 0.78
query15 0.83 0.82 0.82
query16 0.37 0.36 0.38
query17 1.04 1.01 1.00
query18 0.22 0.22 0.23
query19 1.83 1.78 1.77
query20 0.01 0.01 0.01
query21 15.42 0.74 0.65
query22 4.10 6.25 2.72
query23 18.20 1.36 1.19
query24 2.07 0.21 0.23
query25 0.16 0.08 0.08
query26 0.31 0.21 0.22
query27 0.45 0.23 0.23
query28 13.32 1.02 1.00
query29 12.65 3.28 3.28
query30 0.23 0.06 0.05
query31 2.97 0.40 0.37
query32 3.22 0.48 0.48
query33 2.90 2.90 2.92
query34 16.99 4.37 4.36
query35 4.45 4.41 4.38
query36 0.66 0.50 0.48
query37 0.20 0.16 0.16
query38 0.17 0.15 0.14
query39 0.04 0.04 0.05
query40 0.15 0.12 0.12
query41 0.09 0.05 0.04
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 109.63 s
Total hot run time: 31.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2213754620
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2270535054
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2270719410
<details>
<summary>ClickBench: <b>Total hot run time: 30.21 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9cf275204188b55e51bef85d4ca078823f4a5199, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.04
query4 1.67 0.07 0.07
query5 0.48 0.50 0.48
query6 1.14 0.71 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.57 0.51 0.52
query10 0.57 0.56 0.55
query11 0.15 0.12 0.11
query12 0.15 0.12 0.12
query13 0.62 0.59 0.59
query14 0.78 0.79 0.79
query15 0.90 0.87 0.87
query16 0.36 0.36 0.37
query17 0.98 0.99 1.01
query18 0.21 0.22 0.20
query19 1.86 1.73 1.73
query20 0.02 0.01 0.01
query21 15.43 0.76 0.65
query22 3.81 7.12 1.53
query23 18.01 1.32 1.25
query24 2.25 0.22 0.21
query25 0.19 0.08 0.07
query26 0.31 0.22 0.21
query27 0.46 0.23 0.23
query28 13.19 0.99 0.97
query29 12.53 3.40 3.36
query30 0.25 0.06 0.05
query31 2.85 0.41 0.40
query32 3.26 0.50 0.48
query33 2.93 2.93 3.00
query34 15.46 4.25 4.25
query35 4.28 4.31 4.28
query36 0.68 0.48 0.47
query37 0.20 0.16 0.16
query38 0.17 0.15 0.15
query39 0.04 0.04 0.04
query40 0.16 0.13 0.12
query41 0.10 0.05 0.05
query42 0.05 0.05 0.05
query43 0.05 0.05 0.04
Total cold run time: 107.53 s
Total hot run time: 30.21 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287768587
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2272602634
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287810187
<details>
<summary>TPC-H: <b>Total hot run time: 40184 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false
------ Round 1 ----------------------------------
q1 17618 4506 4361 4361
q2 2017 195 174 174
q3 10678 1253 1203 1203
q4 10705 794 760 760
q5 7827 2841 2820 2820
q6 230 140 143 140
q7 995 607 614 607
q8 9399 2079 2118 2079
q9 9335 6723 6739 6723
q10 7465 2245 2185 2185
q11 454 243 249 243
q12 396 229 218 218
q13 18032 3003 2983 2983
q14 289 237 240 237
q15 544 468 505 468
q16 497 406 387 387
q17 999 638 709 638
q18 8047 7420 7327 7327
q19 1563 1014 909 909
q20 689 318 325 318
q21 5349 4404 4503 4404
q22 1109 1023 1000 1000
Total cold run time: 114237 ms
Total hot run time: 40184 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4427 4241 4240 4240
q2 386 280 267 267
q3 2868 2660 2651 2651
q4 1951 1752 1770 1752
q5 5714 5646 5576 5576
q6 226 163 135 135
q7 2080 1684 1765 1684
q8 3302 3502 3424 3424
q9 8846 8838 8718 8718
q10 3460 3263 3265 3263
q11 609 506 498 498
q12 813 612 614 612
q13 16012 3111 3178 3111
q14 323 291 278 278
q15 511 502 494 494
q16 470 431 449 431
q17 1881 1556 1517 1517
q18 8214 7901 7814 7814
q19 5873 1620 1586 1586
q20 2115 1896 1835 1835
q21 5500 5027 5246 5027
q22 1121 1054 1042 1042
Total cold run time: 76702 ms
Total hot run time: 55955 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2217632347
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2217824233
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2259954819
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2268165682
<details>
<summary>TPC-H: <b>Total hot run time: 42173 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f7c0bf4e52d411790a49f49fe80caa97968218e1, data reload: false
------ Round 1 ----------------------------------
q1 18198 4273 4221 4221
q2 2498 214 204 204
q3 11720 1410 1381 1381
q4 10563 902 955 902
q5 7971 3103 3059 3059
q6 227 148 149 148
q7 1101 646 620 620
q8 9447 1859 2005 1859
q9 8554 6623 6658 6623
q10 8716 3886 3858 3858
q11 423 256 252 252
q12 415 236 234 234
q13 17755 2961 2948 2948
q14 271 248 235 235
q15 527 486 488 486
q16 492 416 396 396
q17 979 931 912 912
q18 8199 7299 7227 7227
q19 1398 1237 1231 1231
q20 572 334 342 334
q21 5391 4832 4773 4773
q22 351 299 270 270
Total cold run time: 115768 ms
Total hot run time: 42173 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4083 4039 4061 4039
q2 327 225 219 219
q3 3031 3005 3032 3005
q4 1913 1908 1879 1879
q5 5270 5242 5260 5242
q6 216 132 134 132
q7 2078 1706 1689 1689
q8 3203 3293 3259 3259
q9 8390 8328 8352 8328
q10 3821 3873 3850 3850
q11 563 451 452 451
q12 708 545 560 545
q13 12256 2917 2984 2917
q14 291 255 254 254
q15 518 482 476 476
q16 450 412 401 401
q17 1733 1734 1730 1730
q18 7836 7337 7318 7318
q19 1687 1724 1697 1697
q20 1978 1796 1756 1756
q21 5395 5223 5225 5223
q22 511 442 440 440
Total cold run time: 66258 ms
Total hot run time: 54850 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2216222722
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2217321723
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2217839757
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2260251789
<details>
<summary>TPC-DS: <b>Total hot run time: 171061 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e34d3f2f2e8fc40588dd96971b4ffb5d1411f77, data reload: false
query1 921 389 386 386
query2 6503 1741 1710 1710
query3 6671 222 244 222
query4 19659 17497 17296 17296
query5 3681 533 512 512
query6 305 176 173 173
query7 4600 301 300 300
query8 286 226 229 226
query9 8541 2413 2381 2381
query10 444 293 283 283
query11 10491 10077 10161 10077
query12 127 95 93 93
query13 1647 390 397 390
query14 10413 7289 7812 7289
query15 208 166 170 166
query16 6978 459 457 457
query17 990 589 585 585
query18 1911 310 305 305
query19 207 162 163 162
query20 97 93 91 91
query21 213 108 105 105
query22 4374 3944 3959 3944
query23 33881 33727 33510 33510
query24 10133 3151 3080 3080
query25 695 428 446 428
query26 1637 161 162 161
query27 2988 285 287 285
query28 7484 2011 2010 2010
query29 1218 472 451 451
query30 240 157 162 157
query31 946 795 799 795
query32 107 61 64 61
query33 702 348 338 338
query34 926 489 517 489
query35 904 763 766 763
query36 1031 914 950 914
query37 228 86 84 84
query38 2918 2859 2852 2852
query39 876 849 836 836
query40 255 118 116 116
query41 54 48 48 48
query42 126 101 108 101
query43 449 418 431 418
query44 1163 733 725 725
query45 215 186 183 183
query46 1096 825 796 796
query47 1782 1716 1715 1715
query48 380 302 296 296
query49 954 456 432 432
query50 913 437 437 437
query51 6790 6676 6615 6615
query52 111 94 95 94
query53 258 187 179 179
query54 624 459 467 459
query55 78 75 77 75
query56 295 271 265 265
query57 1133 1031 1076 1031
query58 292 282 316 282
query59 2659 2353 2303 2303
query60 302 295 285 285
query61 118 118 117 117
query62 910 649 678 649
query63 213 187 187 187
query64 5869 2065 2058 2058
query65 3170 3093 3084 3084
query66 1090 367 359 359
query67 15469 14742 14638 14638
query68 4294 565 578 565
query69 466 312 306 306
query70 1115 1079 1063 1063
query71 365 286 277 277
query72 7209 2848 2611 2611
query73 773 336 338 336
query74 6005 5640 5751 5640
query75 3388 2726 2739 2726
query76 2110 1369 1422 1369
query77 484 340 340 340
query78 9513 8995 8948 8948
query79 2732 534 535 534
query80 1357 539 528 528
query81 564 238 235 235
query82 774 132 135 132
query83 208 181 179 179
query84 278 83 86 83
query85 1390 404 360 360
query86 450 300 317 300
query87 3315 3155 3121 3121
query88 3758 2415 2409 2409
query89 398 292 294 292
query90 1772 239 195 195
query91 135 113 116 113
query92 69 57 56 56
query93 2566 623 627 623
query94 767 319 311 311
query95 390 279 284 279
query96 613 276 281 276
query97 3234 3051 3065 3051
query98 226 209 200 200
query99 1631 1289 1320 1289
Total cold run time: 265298 ms
Total hot run time: 171061 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on code in PR #37482:
URL: https://github.com/apache/doris/pull/37482#discussion_r1716244323
##########
be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:
##########
@@ -139,37 +127,44 @@ DorisCompoundReader::DorisCompoundReader(lucene::store::Directory* d, const char
entry->file_name = aid;
entry->offset = stream->readLong();
entry->length = stream->readLong();
- entries->put(aid, entry);
+ _entries->put(aid, entry);
// read header file data
if (entry->offset < 0) {
copyFile(entry->file_name.c_str(), entry->length, buffer, BUFFER_LENGTH);
}
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287734447
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2268175142
<details>
<summary>TPC-DS: <b>Total hot run time: 169382 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f7c0bf4e52d411790a49f49fe80caa97968218e1, data reload: false
query1 920 374 365 365
query2 6489 1703 1649 1649
query3 6678 218 225 218
query4 20368 17461 17385 17385
query5 4290 511 502 502
query6 267 163 167 163
query7 4614 305 287 287
query8 258 204 192 192
query9 8505 2370 2355 2355
query10 429 278 269 269
query11 10617 10109 10222 10109
query12 143 89 87 87
query13 1656 391 377 377
query14 9855 6980 6996 6980
query15 202 173 167 167
query16 7125 466 498 466
query17 1181 586 566 566
query18 2112 290 291 290
query19 200 151 150 150
query20 94 87 91 87
query21 220 103 101 101
query22 4161 3918 3989 3918
query23 33718 32992 33155 32992
query24 10341 3100 2987 2987
query25 703 414 408 408
query26 1767 160 157 157
query27 2761 284 276 276
query28 6836 1966 1959 1959
query29 1358 448 443 443
query30 286 150 152 150
query31 942 773 767 767
query32 109 56 60 56
query33 721 325 327 325
query34 899 478 492 478
query35 851 736 725 725
query36 1009 870 890 870
query37 303 85 82 82
query38 2875 2754 2805 2754
query39 866 810 830 810
query40 296 117 118 117
query41 49 46 46 46
query42 122 105 99 99
query43 489 420 428 420
query44 1198 739 733 733
query45 210 181 181 181
query46 1099 851 810 810
query47 1809 1746 1715 1715
query48 375 296 297 296
query49 1243 443 434 434
query50 942 454 443 443
query51 6841 6682 6738 6682
query52 117 91 190 91
query53 258 184 180 180
query54 640 447 456 447
query55 77 76 75 75
query56 290 253 262 253
query57 1144 1053 1051 1051
query58 273 293 265 265
query59 2595 2576 2356 2356
query60 298 289 280 280
query61 95 93 92 92
query62 930 666 681 666
query63 221 188 191 188
query64 5844 1921 1900 1900
query65 3291 3123 3090 3090
query66 1431 357 333 333
query67 15329 14789 15096 14789
query68 4752 576 589 576
query69 741 382 294 294
query70 1150 1082 1068 1068
query71 503 286 283 283
query72 8009 2666 2545 2545
query73 845 326 332 326
query74 6080 5691 5723 5691
query75 4306 2721 2755 2721
query76 3795 1283 1311 1283
query77 703 318 318 318
query78 9562 8883 8947 8883
query79 2618 544 542 542
query80 1095 510 516 510
query81 556 222 225 222
query82 773 133 132 132
query83 375 171 171 171
query84 264 83 81 81
query85 1503 325 311 311
query86 433 293 295 293
query87 3244 3115 3119 3115
query88 3796 2416 2414 2414
query89 397 290 290 290
query90 2098 197 194 194
query91 132 104 107 104
query92 64 51 56 51
query93 3692 624 622 622
query94 976 312 301 301
query95 378 270 268 268
query96 639 281 280 280
query97 3263 3048 3103 3048
query98 228 205 207 205
query99 1607 1378 1286 1286
Total cold run time: 272812 ms
Total hot run time: 169382 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2262161475
<details>
<summary>TPC-DS: <b>Total hot run time: 169557 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 639d0bea360b4cb4807b38b9d5191d5cc5818713, data reload: false
query1 929 388 381 381
query2 6512 1653 1698 1653
query3 6658 213 219 213
query4 19914 17474 17216 17216
query5 4262 522 531 522
query6 280 182 191 182
query7 4594 304 302 302
query8 276 191 191 191
query9 8543 2375 2361 2361
query10 430 312 278 278
query11 10504 10022 9960 9960
query12 144 88 86 86
query13 1642 386 376 376
query14 9229 7897 7760 7760
query15 225 168 170 168
query16 7106 422 474 422
query17 960 559 532 532
query18 1915 299 284 284
query19 195 146 152 146
query20 92 87 85 85
query21 204 103 97 97
query22 4209 3921 3862 3862
query23 33695 32851 33057 32851
query24 10370 3061 3056 3056
query25 688 390 386 386
query26 1744 156 149 149
query27 2841 278 281 278
query28 6932 1969 1957 1957
query29 1320 437 415 415
query30 289 149 148 148
query31 921 752 742 742
query32 100 57 57 57
query33 701 312 316 312
query34 919 474 493 474
query35 862 755 729 729
query36 994 881 886 881
query37 295 81 83 81
query38 2883 2777 2722 2722
query39 872 804 805 804
query40 283 113 114 113
query41 47 43 44 43
query42 122 102 108 102
query43 461 414 411 411
query44 1219 740 741 740
query45 204 176 176 176
query46 1084 820 793 793
query47 1761 1704 1707 1704
query48 365 294 297 294
query49 1209 432 421 421
query50 893 437 446 437
query51 6853 6763 6695 6695
query52 103 89 92 89
query53 266 185 182 182
query54 640 456 447 447
query55 76 83 76 76
query56 274 266 259 259
query57 1122 1063 1015 1015
query58 297 271 311 271
query59 2525 2478 2412 2412
query60 325 289 342 289
query61 102 95 99 95
query62 925 656 652 652
query63 222 194 196 194
query64 5949 1921 1915 1915
query65 3158 3095 3094 3094
query66 1442 343 335 335
query67 15409 15047 14812 14812
query68 4341 562 590 562
query69 459 300 292 292
query70 1071 1064 1048 1048
query71 414 277 282 277
query72 7088 2678 2509 2509
query73 762 330 329 329
query74 6008 5580 5744 5580
query75 3421 2778 2785 2778
query76 2532 1362 1440 1362
query77 494 320 325 320
query78 9413 8948 8917 8917
query79 1376 569 562 562
query80 1514 522 507 507
query81 568 225 227 225
query82 1036 133 127 127
query83 272 180 166 166
query84 268 85 77 77
query85 1290 320 360 320
query86 419 290 298 290
query87 3230 3121 3138 3121
query88 2992 2418 2399 2399
query89 380 292 290 290
query90 1844 197 200 197
query91 127 103 100 100
query92 62 51 52 51
query93 1417 635 641 635
query94 932 304 272 272
query95 395 265 273 265
query96 591 277 278 277
query97 3215 3036 3076 3036
query98 233 203 192 192
query99 1671 1289 1290 1289
Total cold run time: 262420 ms
Total hot run time: 169557 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2262042083
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2260233662
<details>
<summary>TPC-H: <b>Total hot run time: 41713 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e34d3f2f2e8fc40588dd96971b4ffb5d1411f77, data reload: false
------ Round 1 ----------------------------------
q1 17671 4561 4128 4128
q2 2035 201 200 200
q3 10588 1317 1392 1317
q4 10226 834 941 834
q5 7682 2950 2992 2950
q6 222 137 140 137
q7 1054 620 617 617
q8 9449 1800 1968 1800
q9 8599 6607 6612 6607
q10 8702 3869 3838 3838
q11 432 254 258 254
q12 422 233 232 232
q13 17767 2948 2997 2948
q14 275 256 246 246
q15 533 490 496 490
q16 499 401 390 390
q17 992 948 923 923
q18 8252 7340 7231 7231
q19 1487 1227 1223 1223
q20 560 333 340 333
q21 5325 4730 4745 4730
q22 354 290 285 285
Total cold run time: 113126 ms
Total hot run time: 41713 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4090 4009 4044 4009
q2 332 222 220 220
q3 3017 3071 3175 3071
q4 2069 2019 2017 2017
q5 5671 5506 5458 5458
q6 221 131 131 131
q7 2172 1805 1863 1805
q8 3357 3422 3391 3391
q9 8714 8771 8896 8771
q10 3986 4124 3922 3922
q11 547 474 481 474
q12 773 625 608 608
q13 15002 3119 3160 3119
q14 300 288 273 273
q15 519 503 493 493
q16 495 440 414 414
q17 1773 1771 1785 1771
q18 8235 7810 7723 7723
q19 1854 1723 1757 1723
q20 2118 1840 1842 1840
q21 5804 5535 5267 5267
q22 571 471 463 463
Total cold run time: 71620 ms
Total hot run time: 56963 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2260030790
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2273174788
<details>
<summary>TPC-H: <b>Total hot run time: 42050 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8a3b6b51e28c0fe96d11892b3c267f0618a0168, data reload: false
------ Round 1 ----------------------------------
q1 17656 4303 4116 4116
q2 2031 202 199 199
q3 10547 1403 1361 1361
q4 10237 819 965 819
q5 7675 3019 2989 2989
q6 222 139 138 138
q7 1081 631 625 625
q8 9433 1960 1994 1960
q9 8470 6649 6628 6628
q10 8745 3881 3854 3854
q11 419 247 251 247
q12 409 240 228 228
q13 17765 2942 2933 2933
q14 272 243 245 243
q15 521 488 484 484
q16 496 404 389 389
q17 989 937 942 937
q18 8124 7287 7413 7287
q19 1382 1234 1242 1234
q20 557 328 339 328
q21 5440 4770 4868 4770
q22 361 282 281 281
Total cold run time: 112832 ms
Total hot run time: 42050 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4049 4116 4028 4028
q2 327 217 215 215
q3 3006 3073 3203 3073
q4 2058 2097 2010 2010
q5 5693 5502 5458 5458
q6 228 136 131 131
q7 2155 1848 1774 1774
q8 3331 3336 3400 3336
q9 8661 8726 8732 8726
q10 3930 4036 3981 3981
q11 564 469 464 464
q12 789 630 623 623
q13 16087 3135 3099 3099
q14 305 271 281 271
q15 543 487 484 484
q16 463 411 413 411
q17 1796 1739 1750 1739
q18 8293 7784 7705 7705
q19 1783 1769 1792 1769
q20 2053 1854 1861 1854
q21 5841 5420 5484 5420
q22 521 452 488 452
Total cold run time: 72476 ms
Total hot run time: 57023 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2218047421
<details>
<summary>TPC-H: <b>Total hot run time: 40355 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false
------ Round 1 ----------------------------------
q1 17755 4555 4403 4403
q2 2039 193 198 193
q3 10511 1266 1149 1149
q4 10248 815 788 788
q5 7491 2718 2684 2684
q6 233 139 140 139
q7 975 603 605 603
q8 9233 2147 2124 2124
q9 8799 6567 6562 6562
q10 8965 3772 3767 3767
q11 461 231 239 231
q12 449 231 234 231
q13 17772 2965 3006 2965
q14 267 220 231 220
q15 537 487 478 478
q16 521 376 375 375
q17 976 639 754 639
q18 8256 7567 7450 7450
q19 3598 1536 1446 1446
q20 650 324 311 311
q21 4930 3275 3264 3264
q22 396 333 333 333
Total cold run time: 115062 ms
Total hot run time: 40355 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4450 4320 4296 4296
q2 374 273 269 269
q3 3006 2812 2860 2812
q4 1980 1715 1714 1714
q5 5684 5535 5503 5503
q6 226 134 134 134
q7 2211 1825 1892 1825
q8 3313 3448 3480 3448
q9 8774 8730 8817 8730
q10 4119 3961 3819 3819
q11 592 491 504 491
q12 820 672 627 627
q13 16560 3144 3245 3144
q14 308 280 275 275
q15 534 481 496 481
q16 497 436 428 428
q17 1862 1546 1546 1546
q18 8281 7944 7826 7826
q19 1838 1599 1660 1599
q20 2123 1906 1867 1867
q21 5125 5066 4956 4956
q22 652 548 535 535
Total cold run time: 73329 ms
Total hot run time: 56325 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287768621
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287823229
<details>
<summary>TPC-DS: <b>Total hot run time: 188837 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false
query1 1246 884 863 863
query2 6277 1895 1785 1785
query3 10583 3863 3770 3770
query4 58985 25119 23109 23109
query5 5863 501 518 501
query6 497 164 167 164
query7 6525 300 298 298
query8 319 202 208 202
query9 9301 2402 2364 2364
query10 514 271 267 267
query11 16782 14987 15132 14987
query12 181 113 109 109
query13 1657 403 372 372
query14 12108 6437 7762 6437
query15 240 159 166 159
query16 7669 538 500 500
query17 1116 593 562 562
query18 2026 301 310 301
query19 267 152 147 147
query20 118 113 111 111
query21 215 105 99 99
query22 4579 4289 4182 4182
query23 34222 33396 33128 33128
query24 5584 2873 2828 2828
query25 534 382 371 371
query26 687 165 166 165
query27 1799 271 275 271
query28 3850 2021 2011 2011
query29 684 403 403 403
query30 235 146 145 145
query31 921 728 730 728
query32 89 55 54 54
query33 446 276 270 270
query34 860 468 480 468
query35 811 743 736 736
query36 1031 930 916 916
query37 133 76 79 76
query38 4001 3771 3777 3771
query39 1426 1357 1411 1357
query40 197 118 117 117
query41 47 43 43 43
query42 119 96 99 96
query43 503 455 464 455
query44 1094 739 733 733
query45 188 164 159 159
query46 1085 749 762 749
query47 1852 1744 1755 1744
query48 367 293 304 293
query49 748 411 417 411
query50 804 411 407 407
query51 6771 6742 6695 6695
query52 112 86 88 86
query53 256 185 180 180
query54 544 439 455 439
query55 79 77 75 75
query56 272 251 248 248
query57 1104 1073 1062 1062
query58 225 227 233 227
query59 2897 2718 2921 2718
query60 303 269 279 269
query61 113 114 117 114
query62 742 626 647 626
query63 211 183 181 181
query64 3303 1834 1786 1786
query65 3233 3176 3149 3149
query66 713 349 345 345
query67 15133 14778 14699 14699
query68 5378 560 553 553
query69 431 276 295 276
query70 1190 1148 1133 1133
query71 466 274 280 274
query72 7069 2396 2211 2211
query73 819 334 329 329
query74 9170 8649 8812 8649
query75 3382 2710 2653 2653
query76 2721 999 1122 999
query77 566 309 305 305
query78 10876 9058 10324 9058
query79 9023 541 544 541
query80 2543 495 503 495
query81 594 222 219 219
query82 1324 134 129 129
query83 299 148 143 143
query84 280 79 76 76
query85 1820 277 267 267
query86 383 281 293 281
query87 4545 4206 4233 4206
query88 4775 2558 2475 2475
query89 433 290 291 290
query90 1637 203 196 196
query91 122 97 95 95
query92 60 52 52 52
query93 6014 536 557 536
query94 714 297 286 286
query95 337 308 266 266
query96 628 280 273 273
query97 3178 2990 3061 2990
query98 220 207 211 207
query99 1499 1296 1281 1281
Total cold run time: 329892 ms
Total hot run time: 188837 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2287827890
<details>
<summary>ClickBench: <b>Total hot run time: 31.43 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03ce88db6ce65b8af6bcb221ba63ecb2e642c2ca, data reload: false
query1 0.05 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.65 0.10 0.09
query5 0.53 0.48 0.49
query6 1.13 0.71 0.73
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.54 0.50 0.49
query10 0.53 0.53 0.53
query11 0.16 0.12 0.11
query12 0.14 0.12 0.12
query13 0.59 0.60 0.59
query14 0.76 0.79 0.77
query15 0.86 0.83 0.82
query16 0.37 0.35 0.37
query17 0.96 1.00 0.99
query18 0.23 0.23 0.21
query19 1.94 1.78 1.72
query20 0.01 0.01 0.01
query21 15.39 0.76 0.67
query22 4.97 5.54 2.58
query23 18.35 1.39 1.30
query24 2.10 0.22 0.22
query25 0.15 0.09 0.08
query26 0.29 0.22 0.21
query27 0.46 0.23 0.22
query28 13.30 1.03 1.01
query29 12.60 3.34 3.31
query30 0.24 0.05 0.05
query31 2.90 0.40 0.38
query32 3.24 0.48 0.48
query33 2.94 2.94 2.94
query34 17.05 4.33 4.40
query35 4.42 4.42 4.41
query36 0.66 0.47 0.47
query37 0.19 0.15 0.15
query38 0.15 0.16 0.15
query39 0.05 0.04 0.05
query40 0.16 0.12 0.12
query41 0.08 0.05 0.05
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 110.63 s
Total hot run time: 31.43 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2285199114
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2268103302
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2270685973
<details>
<summary>TPC-H: <b>Total hot run time: 42052 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9cf275204188b55e51bef85d4ca078823f4a5199, data reload: false
------ Round 1 ----------------------------------
q1 17629 4313 4188 4188
q2 2027 201 199 199
q3 10620 1333 1375 1333
q4 10224 905 932 905
q5 7659 3001 2984 2984
q6 225 139 139 139
q7 1064 627 621 621
q8 9446 1920 1990 1920
q9 8357 6602 6587 6587
q10 8697 3836 3853 3836
q11 427 248 250 248
q12 413 234 234 234
q13 17758 2941 2936 2936
q14 268 241 238 238
q15 521 491 489 489
q16 495 385 393 385
q17 984 912 895 895
q18 8014 7356 7187 7187
q19 1848 1233 1220 1220
q20 561 324 354 324
q21 5370 4905 4918 4905
q22 361 280 279 279
Total cold run time: 112968 ms
Total hot run time: 42052 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4103 4024 4037 4024
q2 327 219 220 219
q3 2993 3023 3153 3023
q4 2032 2040 2020 2020
q5 5643 5470 5440 5440
q6 221 133 136 133
q7 2123 1805 1837 1805
q8 3318 3384 3390 3384
q9 8717 8674 8764 8674
q10 3950 4069 3896 3896
q11 564 473 462 462
q12 782 610 598 598
q13 16378 3125 3131 3125
q14 308 266 270 266
q15 537 495 488 488
q16 464 409 422 409
q17 1774 1767 1727 1727
q18 8380 7738 7674 7674
q19 1715 1719 1711 1711
q20 2076 1845 1849 1845
q21 5602 5547 5335 5335
q22 533 493 470 470
Total cold run time: 72540 ms
Total hot run time: 56728 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2270707308
<details>
<summary>TPC-DS: <b>Total hot run time: 170033 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9cf275204188b55e51bef85d4ca078823f4a5199, data reload: false
query1 903 382 370 370
query2 6473 1746 1740 1740
query3 6659 215 221 215
query4 19933 17262 17399 17262
query5 3625 515 522 515
query6 295 160 175 160
query7 4595 300 291 291
query8 255 202 191 191
query9 8528 2336 2324 2324
query10 432 289 265 265
query11 10415 10068 10081 10068
query12 126 91 88 88
query13 1663 380 380 380
query14 9282 7370 7635 7370
query15 198 167 174 167
query16 6851 493 462 462
query17 953 577 559 559
query18 1911 292 288 288
query19 193 149 149 149
query20 98 88 89 88
query21 206 100 99 99
query22 4175 4095 3969 3969
query23 33783 33856 33347 33347
query24 10317 3209 3206 3206
query25 694 397 408 397
query26 1790 154 170 154
query27 2958 295 297 295
query28 7417 1998 1983 1983
query29 1223 459 450 450
query30 254 158 155 155
query31 927 755 761 755
query32 101 60 62 60
query33 692 311 362 311
query34 912 507 524 507
query35 891 769 763 763
query36 1049 879 904 879
query37 282 85 84 84
query38 2948 2854 2815 2815
query39 871 814 821 814
query40 248 114 116 114
query41 46 44 43 43
query42 116 105 99 99
query43 460 412 423 412
query44 1191 732 745 732
query45 219 176 177 176
query46 1091 826 807 807
query47 1812 1730 1708 1708
query48 372 302 300 300
query49 947 426 431 426
query50 898 449 441 441
query51 6811 6737 6687 6687
query52 98 89 91 89
query53 254 184 190 184
query54 641 460 479 460
query55 79 76 77 76
query56 269 259 262 259
query57 1137 1054 1056 1054
query58 256 292 270 270
query59 2608 2297 2300 2297
query60 295 273 284 273
query61 94 123 91 91
query62 872 657 674 657
query63 220 180 189 180
query64 5633 1908 1875 1875
query65 3168 3125 3118 3118
query66 1322 322 332 322
query67 15414 14721 14666 14666
query68 4338 613 572 572
query69 430 309 301 301
query70 1148 1082 1097 1082
query71 368 291 291 291
query72 7150 2720 2478 2478
query73 762 331 334 331
query74 6027 5651 5648 5648
query75 3372 2752 2759 2752
query76 2130 1235 1256 1235
query77 425 318 328 318
query78 9487 8915 8848 8848
query79 2750 531 539 531
query80 1154 528 510 510
query81 564 225 222 222
query82 837 140 136 136
query83 237 173 175 173
query84 258 81 85 81
query85 1405 304 299 299
query86 479 304 288 288
query87 3257 3042 3072 3042
query88 3903 2491 2511 2491
query89 386 291 288 288
query90 1779 191 194 191
query91 123 102 105 102
query92 61 48 51 48
query93 2203 629 631 629
query94 837 302 276 276
query95 387 277 272 272
query96 607 283 279 279
query97 3224 3061 3050 3050
query98 217 195 193 193
query99 1617 1296 1265 1265
Total cold run time: 263371 ms
Total hot run time: 170033 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2273192920
<details>
<summary>TPC-DS: <b>Total hot run time: 168435 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8a3b6b51e28c0fe96d11892b3c267f0618a0168, data reload: false
query1 911 378 377 377
query2 6451 1716 1654 1654
query3 6646 221 224 221
query4 20154 17580 17240 17240
query5 3644 523 506 506
query6 273 181 157 157
query7 4602 303 289 289
query8 242 200 202 200
query9 8514 2399 2396 2396
query10 458 284 263 263
query11 10510 10037 9967 9967
query12 127 88 89 88
query13 1630 389 375 375
query14 8961 6812 6025 6025
query15 204 159 166 159
query16 6979 458 487 458
query17 962 569 552 552
query18 1910 289 292 289
query19 199 155 146 146
query20 90 90 87 87
query21 206 102 110 102
query22 4118 4157 3921 3921
query23 33571 34057 33453 33453
query24 10266 3184 3119 3119
query25 742 420 402 402
query26 1757 157 162 157
query27 3043 294 292 292
query28 7565 2047 2038 2038
query29 1359 449 429 429
query30 232 162 159 159
query31 957 750 757 750
query32 101 54 57 54
query33 676 334 319 319
query34 922 509 506 506
query35 893 782 722 722
query36 1044 886 894 886
query37 295 98 89 89
query38 2922 2820 2772 2772
query39 857 815 806 806
query40 248 112 110 110
query41 48 45 44 44
query42 128 102 106 102
query43 445 425 419 419
query44 1173 728 721 721
query45 209 183 185 183
query46 1079 803 782 782
query47 1820 1758 1756 1756
query48 376 294 291 291
query49 941 417 418 417
query50 893 444 437 437
query51 6786 6795 6675 6675
query52 107 87 88 87
query53 251 180 184 180
query54 636 462 450 450
query55 75 76 74 74
query56 273 263 246 246
query57 1150 1036 1054 1036
query58 250 273 285 273
query59 2475 2339 2325 2325
query60 296 273 264 264
query61 96 89 90 89
query62 879 630 640 630
query63 217 183 177 177
query64 5612 1882 1863 1863
query65 3180 3098 3126 3098
query66 1308 330 332 330
query67 15439 14792 14953 14792
query68 4483 549 575 549
query69 466 305 298 298
query70 1104 1073 1061 1061
query71 433 282 275 275
query72 7525 2667 2540 2540
query73 762 333 329 329
query74 6054 5602 5646 5602
query75 3367 2683 2736 2683
query76 2434 1187 1285 1187
query77 467 322 331 322
query78 9456 8995 8849 8849
query79 1660 533 526 526
query80 952 516 517 516
query81 570 229 230 229
query82 521 131 133 131
query83 261 171 165 165
query84 271 82 79 79
query85 1210 321 357 321
query86 432 320 292 292
query87 3330 3117 3107 3107
query88 3062 2486 2492 2486
query89 390 283 292 283
query90 1727 192 191 191
query91 126 101 103 101
query92 64 51 55 51
query93 1995 632 626 626
query94 844 303 306 303
query95 375 269 276 269
query96 609 295 281 281
query97 3234 3040 3090 3040
query98 215 200 200 200
query99 1597 1287 1267 1267
Total cold run time: 261379 ms
Total hot run time: 168435 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #37482:
URL: https://github.com/apache/doris/pull/37482#discussion_r1698225944
##########
be/src/olap/rowset/segment_v2/inverted_index_file_writer.h:
##########
@@ -19,6 +19,7 @@
#include <CLucene.h> // IWYU pragma: keep
Review Comment:
warning: 'CLucene.h' file not found [clang-diagnostic-error]
```cpp
#include <CLucene.h> // IWYU pragma: keep
^
```
##########
be/src/olap/rowset/beta_rowset_writer.h:
##########
@@ -18,6 +18,7 @@
#pragma once
#include <fmt/format.h>
Review Comment:
warning: 'fmt/format.h' file not found [clang-diagnostic-error]
```cpp
#include <fmt/format.h>
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2218069536
<details>
<summary>TPC-DS: <b>Total hot run time: 172656 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false
query1 923 379 372 372
query2 6451 2470 2292 2292
query3 6632 206 213 206
query4 28581 17477 17372 17372
query5 3687 494 493 493
query6 266 195 161 161
query7 4579 284 284 284
query8 328 283 285 283
query9 8465 2421 2390 2390
query10 443 297 274 274
query11 11761 10046 10059 10046
query12 122 80 81 80
query13 1639 387 372 372
query14 10146 7520 6910 6910
query15 240 183 196 183
query16 7405 306 306 306
query17 1386 546 524 524
query18 1667 278 286 278
query19 188 151 156 151
query20 90 81 81 81
query21 215 122 122 122
query22 4366 4134 4092 4092
query23 34013 33534 33641 33534
query24 11361 2931 2917 2917
query25 628 394 411 394
query26 773 146 148 146
query27 2348 280 283 280
query28 7054 2173 2187 2173
query29 918 626 645 626
query30 264 147 156 147
query31 970 768 754 754
query32 95 64 57 57
query33 756 289 337 289
query34 1017 497 507 497
query35 691 575 557 557
query36 1111 960 955 955
query37 153 88 83 83
query38 2907 2906 2894 2894
query39 906 846 848 846
query40 227 149 129 129
query41 59 52 50 50
query42 120 102 106 102
query43 584 529 546 529
query44 1237 732 725 725
query45 191 161 160 160
query46 1067 743 721 721
query47 1845 1755 1777 1755
query48 359 297 299 297
query49 845 411 422 411
query50 776 397 396 396
query51 6938 6866 6814 6814
query52 113 96 96 96
query53 360 288 303 288
query54 927 453 454 453
query55 74 76 76 76
query56 311 276 275 275
query57 1115 1085 1074 1074
query58 246 256 261 256
query59 3503 3022 3293 3022
query60 304 278 313 278
query61 100 94 98 94
query62 795 651 663 651
query63 332 300 286 286
query64 9197 2162 1626 1626
query65 3192 3065 3103 3065
query66 778 327 326 326
query67 15383 14941 15036 14941
query68 4471 559 539 539
query69 504 343 343 343
query70 1197 1120 1170 1120
query71 421 284 292 284
query72 7062 5218 4137 4137
query73 738 332 324 324
query74 6066 5589 5617 5589
query75 3425 2714 2662 2662
query76 2652 958 1031 958
query77 483 311 300 300
query78 9474 9140 8876 8876
query79 2712 519 517 517
query80 1771 477 491 477
query81 580 220 220 220
query82 785 136 136 136
query83 307 175 169 169
query84 278 88 86 86
query85 1948 318 312 312
query86 478 319 312 312
query87 3292 3107 3110 3107
query88 4337 2383 2345 2345
query89 489 392 393 392
query90 1783 195 193 193
query91 130 102 103 102
query92 60 53 51 51
query93 3886 533 531 531
query94 1130 214 219 214
query95 398 313 327 313
query96 612 279 272 272
query97 3159 3007 3032 3007
query98 278 195 197 195
query99 1748 1256 1256 1256
Total cold run time: 283177 ms
Total hot run time: 172656 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2218080760
<details>
<summary>ClickBench: <b>Total hot run time: 30.45 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 14dd4e2fa0113b25e6ff8426789233f7548b589b, data reload: false
query1 0.05 0.03 0.03
query2 0.08 0.04 0.04
query3 0.24 0.05 0.05
query4 1.67 0.08 0.08
query5 0.48 0.47 0.48
query6 1.14 0.73 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.56 0.50 0.48
query10 0.55 0.55 0.54
query11 0.16 0.12 0.12
query12 0.15 0.12 0.13
query13 0.60 0.59 0.58
query14 0.79 0.78 0.77
query15 0.84 0.81 0.81
query16 0.34 0.36 0.36
query17 0.98 1.00 0.94
query18 0.24 0.22 0.22
query19 1.79 1.71 1.69
query20 0.01 0.01 0.01
query21 15.41 0.79 0.66
query22 4.28 6.97 1.83
query23 18.29 1.31 1.24
query24 2.09 0.23 0.24
query25 0.17 0.09 0.08
query26 0.32 0.21 0.20
query27 0.45 0.23 0.23
query28 13.25 1.03 1.00
query29 12.56 3.27 3.28
query30 0.27 0.06 0.06
query31 2.84 0.39 0.39
query32 3.27 0.49 0.47
query33 2.90 2.94 2.85
query34 17.01 4.44 4.37
query35 4.44 4.43 4.52
query36 0.65 0.49 0.47
query37 0.20 0.17 0.15
query38 0.16 0.14 0.14
query39 0.04 0.03 0.04
query40 0.15 0.12 0.13
query41 0.10 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.03
Total cold run time: 109.69 s
Total hot run time: 30.45 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2273202850
<details>
<summary>ClickBench: <b>Total hot run time: 30.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8a3b6b51e28c0fe96d11892b3c267f0618a0168, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.68 0.07 0.08
query5 0.50 0.49 0.48
query6 1.15 0.74 0.72
query7 0.02 0.01 0.01
query8 0.05 0.05 0.05
query9 0.56 0.52 0.55
query10 0.56 0.56 0.57
query11 0.16 0.11 0.11
query12 0.15 0.12 0.13
query13 0.63 0.60 0.59
query14 0.77 0.80 0.80
query15 0.91 0.87 0.87
query16 0.35 0.36 0.36
query17 1.04 1.03 1.00
query18 0.22 0.21 0.21
query19 1.86 1.72 1.77
query20 0.01 0.01 0.01
query21 15.45 0.76 0.65
query22 3.73 8.47 1.49
query23 18.18 1.38 1.26
query24 2.23 0.23 0.22
query25 0.18 0.08 0.08
query26 0.32 0.20 0.22
query27 0.46 0.24 0.23
query28 13.16 0.99 0.97
query29 12.57 3.31 3.27
query30 0.25 0.07 0.06
query31 2.85 0.41 0.42
query32 3.23 0.50 0.49
query33 2.97 2.99 2.95
query34 15.45 4.30 4.30
query35 4.35 4.28 4.25
query36 0.68 0.51 0.48
query37 0.18 0.16 0.16
query38 0.16 0.15 0.15
query39 0.04 0.04 0.04
query40 0.17 0.13 0.14
query41 0.09 0.05 0.05
query42 0.05 0.04 0.05
query43 0.05 0.04 0.04
Total cold run time: 107.76 s
Total hot run time: 30.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2213914282
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "csun5285 (via GitHub)" <gi...@apache.org>.
csun5285 commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2273012491
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye merged PR #37482:
URL: https://github.com/apache/doris/pull/37482
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on code in PR #37482:
URL: https://github.com/apache/doris/pull/37482#discussion_r1673301847
##########
be/src/olap/rowset/segment_v2/inverted_index_file_reader.h:
##########
@@ -65,20 +66,23 @@ class InvertedIndexFileReader {
Status index_file_exist(const TabletIndex* index_meta, bool* res) const;
Status has_null(const TabletIndex* index_meta, bool* res) const;
Result<InvertedIndexDirectoryMap> get_all_directories();
+ // open file v2, init _stream
+ int64_t get_inverted_file_size() const { return _stream == nullptr ? 0 : _stream->length(); }
Review Comment:
delete it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2268180201
<details>
<summary>ClickBench: <b>Total hot run time: 30.16 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f7c0bf4e52d411790a49f49fe80caa97968218e1, data reload: false
query1 0.04 0.04 0.05
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.49 0.48 0.50
query6 1.14 0.73 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.57 0.52 0.52
query10 0.56 0.56 0.58
query11 0.16 0.12 0.11
query12 0.15 0.12 0.13
query13 0.62 0.61 0.60
query14 0.77 0.79 0.80
query15 0.90 0.87 0.87
query16 0.36 0.35 0.35
query17 0.98 1.02 1.01
query18 0.21 0.20 0.20
query19 1.88 1.78 1.79
query20 0.01 0.01 0.00
query21 15.38 0.77 0.67
query22 3.93 8.63 1.21
query23 17.78 1.37 1.34
query24 2.26 0.23 0.21
query25 0.19 0.08 0.08
query26 0.32 0.21 0.21
query27 0.45 0.24 0.24
query28 13.15 0.98 0.96
query29 12.51 3.32 3.30
query30 0.25 0.06 0.05
query31 2.90 0.41 0.41
query32 3.22 0.48 0.49
query33 3.00 2.99 2.98
query34 15.43 4.28 4.28
query35 4.31 4.31 4.29
query36 0.68 0.48 0.48
query37 0.19 0.18 0.16
query38 0.15 0.15 0.15
query39 0.04 0.04 0.04
query40 0.16 0.13 0.13
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 107.38 s
Total hot run time: 30.16 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt] (inverted index) add inverted index file size for open file [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37482:
URL: https://github.com/apache/doris/pull/37482#issuecomment-2285288671
<details>
<summary>TPC-DS: <b>Total hot run time: 202800 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 298c058a1574a85547d95d7d3455b40054c2ae93, data reload: false
query1 1683 1464 1427 1427
query2 6339 1959 1914 1914
query3 14006 8551 312 312
query4 33816 23277 23008 23008
query5 4484 485 480 480
query6 260 161 151 151
query7 3940 291 297 291
query8 247 218 203 203
query9 8410 2468 2431 2431
query10 553 454 451 451
query11 17156 15062 14895 14895
query12 143 96 93 93
query13 1541 363 355 355
query14 9140 7743 7769 7743
query15 289 212 240 212
query16 7674 470 546 470
query17 1486 579 557 557
query18 1977 295 297 295
query19 217 142 149 142
query20 124 110 112 110
query21 217 106 110 106
query22 4562 4288 4249 4249
query23 34326 33597 33555 33555
query24 10465 2607 2626 2607
query25 597 393 388 388
query26 706 154 155 154
query27 2163 281 281 281
query28 6109 2042 2027 2027
query29 814 446 402 402
query30 255 148 151 148
query31 969 754 691 691
query32 94 52 55 52
query33 629 279 270 270
query34 840 468 471 468
query35 944 821 808 808
query36 1065 930 905 905
query37 130 82 88 82
query38 4291 4168 4104 4104
query39 1403 1359 1390 1359
query40 199 117 116 116
query41 45 44 44 44
query42 113 94 99 94
query43 524 460 462 460
query44 1048 719 720 719
query45 243 206 210 206
query46 1077 744 741 741
query47 1820 1737 1785 1737
query48 369 288 296 288
query49 834 410 422 410
query50 801 398 400 398
query51 6804 6680 6734 6680
query52 109 86 92 86
query53 254 181 180 180
query54 915 450 436 436
query55 74 73 79 73
query56 263 244 238 238
query57 1147 1086 1082 1082
query58 219 228 222 222
query59 3019 2673 2898 2673
query60 305 327 263 263
query61 97 94 93 93
query62 794 643 644 643
query63 211 186 183 183
query64 9165 2251 1764 1764
query65 3365 3217 3165 3165
query66 762 341 328 328
query67 15242 14954 14796 14796
query68 5978 549 544 544
query69 430 429 390 390
query70 1164 1045 1151 1045
query71 486 276 268 268
query72 19438 16934 16856 16856
query73 791 333 325 325
query74 9311 8898 8701 8701
query75 3652 2644 2696 2644
query76 3302 1046 956 956
query77 645 317 332 317
query78 9779 9238 8945 8945
query79 4064 520 511 511
query80 2273 489 541 489
query81 600 227 233 227
query82 813 135 134 134
query83 259 149 151 149
query84 278 76 78 76
query85 1278 277 271 271
query86 386 285 301 285
query87 4746 4541 4441 4441
query88 4610 2472 2485 2472
query89 563 293 285 285
query90 1952 197 195 195
query91 124 96 94 94
query92 59 51 48 48
query93 6168 546 535 535
query94 773 306 306 306
query95 354 259 313 259
query96 614 275 272 272
query97 3198 3021 3033 3021
query98 221 202 194 194
query99 1681 1231 1253 1231
Total cold run time: 323872 ms
Total hot run time: 202800 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org