You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2024/09/09 10:23:15 UTC
[PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Mryange opened a new pull request, #40559:
URL: https://github.com/apache/doris/pull/40559
## Proposed changes
```
mysql [test]>SELECT count() from hits_10m WHERE SearchPhrase IN (SELECT SearchPhrase from hits_10m);
+----------+
| count(*) |
+----------+
| 10000000 |
+----------+
1 row in set (1.15 sec)
now
mysql [test]>SELECT count() from hits_10m WHERE SearchPhrase IN (SELECT SearchPhrase from hits_10m);
+----------+
| count(*) |
+----------+
| 9999936 |
+----------+
1 row in set (0.67 sec)
```
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2339891698
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340707521
<details>
<summary>TPC-H: <b>Total hot run time: 38710 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9792c23dbcd67c71efde6e320dc0cf35ea52e065, data reload: false
------ Round 1 ----------------------------------
q1 17673 4538 4422 4422
q2 2037 222 187 187
q3 11320 1005 1146 1005
q4 10518 736 744 736
q5 7743 2860 2865 2860
q6 235 145 141 141
q7 986 621 637 621
q8 9337 2114 2088 2088
q9 7266 6615 6628 6615
q10 7044 2294 2146 2146
q11 464 252 253 252
q12 404 227 222 222
q13 17765 3120 3155 3120
q14 282 246 250 246
q15 547 515 493 493
q16 548 456 445 445
q17 1001 757 746 746
q18 7489 7046 6879 6879
q19 1380 1083 1064 1064
q20 688 353 345 345
q21 3987 3093 3057 3057
q22 1100 1020 1034 1020
Total cold run time: 109814 ms
Total hot run time: 38710 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4417 4353 4301 4301
q2 393 270 279 270
q3 2937 2687 2629 2629
q4 1948 1620 1678 1620
q5 5767 5707 5807 5707
q6 235 137 140 137
q7 2226 1884 1843 1843
q8 3289 3477 3534 3477
q9 8912 8891 8876 8876
q10 3634 3409 3404 3404
q11 634 502 518 502
q12 832 671 721 671
q13 15565 3375 3378 3375
q14 322 303 313 303
q15 536 501 501 501
q16 568 494 534 494
q17 1859 1551 1578 1551
q18 8182 8039 7998 7998
q19 1725 1617 1525 1525
q20 2142 1930 1894 1894
q21 5704 5575 5450 5450
q22 1146 1076 1067 1067
Total cold run time: 72973 ms
Total hot run time: 57595 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340946460
TeamCity be ut coverage result:
Function Coverage: 36.83% (9396/25512)
Line Coverage: 28.23% (77520/274578)
Region Coverage: 27.63% (40014/144826)
Branch Coverage: 24.25% (20348/83918)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6e498ecd9475f866d6935dfc8287810d8815be19_6e498ecd9475f866d6935dfc8287810d8815be19/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #40559:
URL: https://github.com/apache/doris/pull/40559#discussion_r1751217843
##########
be/src/vec/common/hash_table/hash_map_context.h:
##########
@@ -284,29 +285,59 @@ struct MethodStringNoCache : public MethodBase<TData> {
using State =
ColumnsHashing::HashMethodString<typename Base::Value, typename Base::Mapped, true>;
- std::vector<StringRef> stored_keys;
+ // need keep until the hash probe end.
+ std::vector<StringRef> _build_stored_keys;
+ // refresh each time probe
+ std::vector<StringRef> _stored_keys;
size_t serialized_keys_size(bool is_build) const override {
- return stored_keys.size() * sizeof(StringRef);
+ return is_build ? (_build_stored_keys.size() * sizeof(StringRef))
+ : (_stored_keys.size() * sizeof(StringRef));
}
- void init_serialized_keys(const ColumnRawPtrs& key_columns, size_t num_rows,
- const uint8_t* null_map = nullptr, bool is_join = false,
- bool is_build = false, uint32_t bucket_size = 0) override {
- const IColumn& column = *key_columns[0];
- const auto& column_string = assert_cast<const ColumnString&>(
- column.is_nullable()
- ? assert_cast<const ColumnNullable&>(column).get_nested_column()
- : column);
+ void serialized_str_32(const ColumnString& column_string, std::vector<StringRef>& stored_keys) {
Review Comment:
use lambda to do the work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340868305
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2338161951
<details>
<summary>TPC-DS: <b>Total hot run time: 191969 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit adb4e0f84651db6193b6cbc442c680f0c8d222a8, data reload: false
query1 1254 879 894 879
query2 6356 1871 1901 1871
query3 10913 3905 3898 3898
query4 59438 23754 23337 23337
query5 5435 493 506 493
query6 401 170 156 156
query7 5777 295 292 292
query8 310 217 214 214
query9 8773 2495 2490 2490
query10 473 268 272 268
query11 18489 15144 15226 15144
query12 160 104 107 104
query13 1586 394 382 382
query14 10694 7583 7120 7120
query15 238 169 189 169
query16 7523 480 480 480
query17 1140 558 556 556
query18 1968 307 300 300
query19 289 147 147 147
query20 120 106 109 106
query21 203 106 101 101
query22 4543 4484 4424 4424
query23 34546 33361 33256 33256
query24 5935 2891 2851 2851
query25 512 379 391 379
query26 692 153 153 153
query27 1777 268 276 268
query28 3657 2047 2035 2035
query29 691 404 398 398
query30 240 152 152 152
query31 958 768 786 768
query32 85 85 52 52
query33 429 289 280 280
query34 883 467 466 466
query35 855 726 705 705
query36 1084 935 940 935
query37 141 90 82 82
query38 4044 3957 3772 3772
query39 1442 1388 1365 1365
query40 196 113 112 112
query41 45 48 44 44
query42 120 97 96 96
query43 490 466 462 462
query44 1099 746 746 746
query45 195 167 162 162
query46 1077 747 733 733
query47 1851 1797 1804 1797
query48 359 283 291 283
query49 759 439 447 439
query50 842 409 425 409
query51 7011 6950 6944 6944
query52 99 84 87 84
query53 254 178 181 178
query54 553 439 447 439
query55 77 76 72 72
query56 269 248 258 248
query57 1222 1064 1095 1064
query58 232 245 242 242
query59 2925 2801 2642 2642
query60 302 268 264 264
query61 103 101 99 99
query62 714 642 647 642
query63 221 186 190 186
query64 2854 770 755 755
query65 3308 3135 3156 3135
query66 645 359 343 343
query67 15579 15026 15125 15026
query68 2979 586 583 583
query69 387 282 286 282
query70 1168 1133 1150 1133
query71 356 277 282 277
query72 6115 4129 4053 4053
query73 746 323 324 323
query74 9033 8849 8795 8795
query75 3359 2728 2666 2666
query76 1399 1015 983 983
query77 559 326 321 321
query78 9623 8996 9038 8996
query79 1035 550 539 539
query80 708 500 552 500
query81 462 235 232 232
query82 226 135 137 135
query83 167 150 157 150
query84 256 78 72 72
query85 674 283 282 282
query86 300 270 283 270
query87 4510 4366 4213 4213
query88 3118 2287 2264 2264
query89 379 282 279 279
query90 1965 189 187 187
query91 121 95 97 95
query92 61 50 49 49
query93 1063 531 543 531
query94 695 300 304 300
query95 325 248 253 248
query96 582 265 257 257
query97 3189 3046 3031 3031
query98 216 211 198 198
query99 1706 1298 1231 1231
Total cold run time: 304937 ms
Total hot run time: 191969 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2342678252
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2341042075
<details>
<summary>ClickBench: <b>Total hot run time: 31.92 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6e498ecd9475f866d6935dfc8287810d8815be19, data reload: false
query1 0.05 0.05 0.04
query2 0.07 0.04 0.03
query3 0.23 0.06 0.04
query4 1.67 0.08 0.07
query5 0.51 0.51 0.49
query6 1.13 0.74 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.55 0.49 0.50
query10 0.55 0.56 0.53
query11 0.16 0.11 0.11
query12 0.15 0.13 0.12
query13 0.60 0.58 0.59
query14 1.41 1.44 1.42
query15 0.84 0.83 0.82
query16 0.38 0.38 0.39
query17 1.01 1.04 1.06
query18 0.21 0.20 0.20
query19 1.96 1.88 1.83
query20 0.01 0.02 0.01
query21 15.41 0.67 0.65
query22 4.24 7.81 2.26
query23 18.29 1.46 1.26
query24 2.08 0.22 0.20
query25 0.15 0.08 0.08
query26 0.26 0.19 0.18
query27 0.08 0.08 0.07
query28 13.28 1.02 1.01
query29 12.57 3.34 3.34
query30 0.24 0.06 0.05
query31 2.88 0.39 0.40
query32 3.25 0.48 0.48
query33 3.02 2.95 3.04
query34 17.12 4.47 4.52
query35 4.56 4.51 4.52
query36 0.66 0.46 0.47
query37 0.18 0.15 0.15
query38 0.16 0.15 0.14
query39 0.04 0.04 0.03
query40 0.15 0.14 0.13
query41 0.09 0.04 0.04
query42 0.06 0.04 0.05
query43 0.05 0.04 0.04
Total cold run time: 110.38 s
Total hot run time: 31.92 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2338175328
<details>
<summary>ClickBench: <b>Total hot run time: 32.73 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit adb4e0f84651db6193b6cbc442c680f0c8d222a8, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.04 0.04
query3 0.22 0.05 0.06
query4 1.66 0.10 0.09
query5 0.50 0.50 0.50
query6 1.14 0.74 0.73
query7 0.01 0.01 0.02
query8 0.06 0.04 0.05
query9 0.54 0.50 0.49
query10 0.53 0.56 0.55
query11 0.16 0.11 0.11
query12 0.14 0.13 0.12
query13 0.61 0.59 0.59
query14 1.41 1.43 1.41
query15 0.84 0.82 0.82
query16 0.37 0.36 0.37
query17 0.99 1.00 1.04
query18 0.20 0.20 0.20
query19 1.87 1.76 1.74
query20 0.01 0.01 0.01
query21 15.41 0.68 0.67
query22 4.48 5.96 3.23
query23 18.31 1.47 1.27
query24 2.05 0.23 0.21
query25 0.15 0.08 0.08
query26 0.27 0.17 0.18
query27 0.08 0.08 0.08
query28 13.29 1.02 1.00
query29 12.64 3.35 3.35
query30 0.24 0.06 0.05
query31 2.89 0.41 0.41
query32 3.22 0.48 0.48
query33 2.95 3.00 3.01
query34 17.04 4.43 4.35
query35 4.47 4.45 4.39
query36 0.65 0.51 0.50
query37 0.19 0.15 0.15
query38 0.16 0.14 0.15
query39 0.06 0.04 0.04
query40 0.15 0.12 0.12
query41 0.10 0.06 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.29 s
Total hot run time: 32.73 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2337793101
TeamCity be ut coverage result:
Function Coverage: 36.83% (9384/25478)
Line Coverage: 28.23% (77405/274181)
Region Coverage: 27.63% (39965/144650)
Branch Coverage: 24.26% (20337/83820)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4c3a4e09f8e4941cac8029a1c929e61000cf5fc9_4c3a4e09f8e4941cac8029a1c929e61000cf5fc9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2337995348
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2338118782
TeamCity be ut coverage result:
Function Coverage: 36.83% (9385/25481)
Line Coverage: 28.22% (77375/274203)
Region Coverage: 27.63% (39965/144665)
Branch Coverage: 24.25% (20332/83828)
Coverage Report: http://coverage.selectdb-in.cc/coverage/adb4e0f84651db6193b6cbc442c680f0c8d222a8_adb4e0f84651db6193b6cbc442c680f0c8d222a8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2341013384
<details>
<summary>TPC-DS: <b>Total hot run time: 198239 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6e498ecd9475f866d6935dfc8287810d8815be19, data reload: false
query1 1240 884 846 846
query2 6375 2023 1940 1940
query3 10756 4186 4040 4040
query4 59713 24578 23252 23252
query5 5212 508 498 498
query6 398 172 163 163
query7 5605 298 312 298
query8 309 213 210 210
query9 7567 2488 2492 2488
query10 438 281 260 260
query11 17280 15141 15300 15141
query12 160 105 106 105
query13 1458 389 419 389
query14 9657 7458 7445 7445
query15 215 183 188 183
query16 6827 489 497 489
query17 1105 607 565 565
query18 1496 288 299 288
query19 202 150 149 149
query20 125 110 108 108
query21 208 101 105 101
query22 4566 4687 4777 4687
query23 34408 33522 33533 33522
query24 6080 2904 2913 2904
query25 512 378 393 378
query26 610 155 152 152
query27 1581 273 280 273
query28 3559 2032 2014 2014
query29 650 405 407 405
query30 225 156 147 147
query31 913 762 784 762
query32 75 52 56 52
query33 385 289 281 281
query34 877 477 466 466
query35 844 720 711 711
query36 1080 947 916 916
query37 140 89 89 89
query38 4019 3853 3906 3853
query39 1464 1403 1405 1403
query40 207 117 115 115
query41 48 49 47 47
query42 122 95 97 95
query43 522 469 488 469
query44 1114 769 752 752
query45 197 166 168 166
query46 1102 734 742 734
query47 1903 1765 1829 1765
query48 375 336 296 296
query49 768 430 439 430
query50 817 415 408 408
query51 7044 6899 6921 6899
query52 99 87 92 87
query53 256 183 183 183
query54 580 454 462 454
query55 79 75 73 73
query56 285 259 266 259
query57 1181 1112 1084 1084
query58 239 223 232 223
query59 3121 2949 2821 2821
query60 294 273 281 273
query61 104 107 101 101
query62 749 634 632 632
query63 228 218 187 187
query64 1365 692 671 671
query65 3240 3178 3178 3178
query66 681 331 345 331
query67 15892 15516 15401 15401
query68 1189 869 859 859
query69 424 329 329 329
query70 1115 1175 1068 1068
query71 336 350 348 348
query72 4625 3482 3463 3463
query73 581 581 577 577
query74 9003 8944 8885 8885
query75 3146 2922 3033 2922
query76 951 860 851 851
query77 409 410 397 397
query78 9396 9317 9218 9218
query79 898 888 865 865
query80 821 807 806 806
query81 454 255 261 255
query82 268 262 265 262
query83 190 188 191 188
query84 196 105 107 105
query85 580 398 391 391
query86 314 307 329 307
query87 4306 4410 4245 4245
query88 4232 4104 4094 4094
query89 372 369 370 369
query90 796 310 304 304
query91 124 132 139 132
query92 77 78 80 78
query93 926 927 938 927
query94 406 398 446 398
query95 423 412 411 411
query96 471 474 471 471
query97 3103 3129 3106 3106
query98 229 235 229 229
query99 1287 1277 1272 1272
Total cold run time: 292594 ms
Total hot run time: 198239 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340972720
<details>
<summary>TPC-H: <b>Total hot run time: 38212 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6e498ecd9475f866d6935dfc8287810d8815be19, data reload: false
------ Round 1 ----------------------------------
q1 17618 4400 4325 4325
q2 2017 194 183 183
q3 11801 962 1139 962
q4 10512 756 713 713
q5 7739 2868 2804 2804
q6 230 140 141 140
q7 957 639 614 614
q8 9324 2088 2103 2088
q9 7027 6582 6567 6567
q10 7017 2202 2233 2202
q11 462 244 243 243
q12 403 223 226 223
q13 17774 3088 3065 3065
q14 291 233 242 233
q15 538 496 493 493
q16 527 424 431 424
q17 988 639 671 639
q18 7293 6991 6847 6847
q19 1388 1063 1031 1031
q20 656 347 343 343
q21 3988 3133 3034 3034
q22 1148 1062 1039 1039
Total cold run time: 109698 ms
Total hot run time: 38212 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4384 4293 4294 4293
q2 383 274 277 274
q3 2891 2698 2672 2672
q4 1997 1623 1665 1623
q5 5640 5731 5765 5731
q6 235 139 147 139
q7 2243 1861 1871 1861
q8 3298 3512 3464 3464
q9 8947 8871 8930 8871
q10 3596 3388 3433 3388
q11 625 545 517 517
q12 837 659 647 647
q13 15605 3205 3307 3205
q14 304 290 282 282
q15 536 497 514 497
q16 528 510 496 496
q17 1836 1549 1564 1549
q18 8246 7865 7836 7836
q19 1821 1771 1538 1538
q20 2192 1901 1904 1901
q21 5761 5483 5534 5483
q22 1148 1085 1055 1055
Total cold run time: 73053 ms
Total hot run time: 57322 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340773430
<details>
<summary>ClickBench: <b>Total hot run time: 31.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9792c23dbcd67c71efde6e320dc0cf35ea52e065, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.66 0.08 0.08
query5 0.49 0.49 0.50
query6 1.13 0.75 0.73
query7 0.02 0.01 0.02
query8 0.06 0.04 0.04
query9 0.54 0.50 0.50
query10 0.54 0.56 0.55
query11 0.16 0.11 0.11
query12 0.14 0.13 0.12
query13 0.61 0.59 0.59
query14 1.39 1.43 1.43
query15 0.84 0.82 0.87
query16 0.38 0.38 0.38
query17 1.06 1.01 0.99
query18 0.21 0.21 0.20
query19 1.94 1.80 1.74
query20 0.01 0.01 0.02
query21 15.40 0.66 0.66
query22 3.62 7.25 2.02
query23 18.30 1.39 1.28
query24 2.09 0.24 0.21
query25 0.14 0.08 0.08
query26 0.25 0.18 0.17
query27 0.07 0.08 0.07
query28 13.28 1.01 0.99
query29 12.63 3.33 3.29
query30 0.25 0.06 0.06
query31 2.86 0.40 0.40
query32 3.24 0.47 0.47
query33 3.00 2.98 3.01
query34 17.22 4.40 4.48
query35 4.45 4.39 4.43
query36 0.66 0.47 0.48
query37 0.18 0.15 0.16
query38 0.16 0.15 0.14
query39 0.05 0.04 0.04
query40 0.16 0.13 0.13
query41 0.09 0.05 0.05
query42 0.05 0.04 0.05
query43 0.04 0.05 0.05
Total cold run time: 109.71 s
Total hot run time: 31.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2337730447
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2337730492
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2337804660
<details>
<summary>TPC-H: <b>Total hot run time: 37995 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c3a4e09f8e4941cac8029a1c929e61000cf5fc9, data reload: false
------ Round 1 ----------------------------------
q1 17598 4462 4345 4345
q2 2018 200 192 192
q3 11556 936 1109 936
q4 10516 733 725 725
q5 7759 2906 2812 2812
q6 224 143 137 137
q7 972 609 606 606
q8 9344 2070 2028 2028
q9 7158 6489 6557 6489
q10 6997 2237 2223 2223
q11 455 248 245 245
q12 388 228 234 228
q13 18953 3063 3071 3063
q14 295 236 254 236
q15 538 505 471 471
q16 518 423 437 423
q17 968 712 762 712
q18 7423 6917 6814 6814
q19 1383 1014 1121 1014
q20 663 333 352 333
q21 3851 2988 2962 2962
q22 1104 1008 1001 1001
Total cold run time: 110681 ms
Total hot run time: 37995 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4383 4248 4259 4248
q2 377 266 258 258
q3 2886 2648 2721 2648
q4 1997 1703 1616 1616
q5 5694 5668 5792 5668
q6 230 138 143 138
q7 2282 1892 1878 1878
q8 3259 3453 3572 3453
q9 8811 8789 8772 8772
q10 3643 3374 3358 3358
q11 606 542 510 510
q12 844 663 652 652
q13 14302 3181 3260 3181
q14 325 287 283 283
q15 528 491 496 491
q16 521 491 501 491
q17 1822 1535 1536 1535
q18 8290 7795 7822 7795
q19 1755 1570 1567 1567
q20 2146 1945 1896 1896
q21 5751 5460 5543 5460
q22 1112 1015 1010 1010
Total cold run time: 71564 ms
Total hot run time: 56908 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340753311
<details>
<summary>TPC-DS: <b>Total hot run time: 198262 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9792c23dbcd67c71efde6e320dc0cf35ea52e065, data reload: false
query1 1268 895 877 877
query2 6367 2012 1990 1990
query3 10657 4001 3970 3970
query4 60342 25942 23189 23189
query5 4963 500 502 500
query6 393 178 165 165
query7 5633 299 293 293
query8 313 216 210 210
query9 7643 2521 2476 2476
query10 413 269 264 264
query11 16873 15067 15164 15067
query12 158 105 110 105
query13 1471 407 434 407
query14 10306 7587 7775 7587
query15 212 182 178 178
query16 6788 492 468 468
query17 1062 556 559 556
query18 1636 307 304 304
query19 208 144 148 144
query20 123 114 115 114
query21 220 110 112 110
query22 4487 4388 4511 4388
query23 35382 33742 33761 33742
query24 5950 2888 2814 2814
query25 519 403 405 403
query26 612 158 164 158
query27 1605 291 285 285
query28 3883 2071 2049 2049
query29 678 436 440 436
query30 234 161 153 153
query31 934 743 758 743
query32 75 58 59 58
query33 471 296 287 287
query34 879 477 474 474
query35 839 733 731 731
query36 1062 905 940 905
query37 196 83 84 83
query38 4004 4030 3881 3881
query39 1430 1400 1387 1387
query40 202 113 108 108
query41 44 48 43 43
query42 115 96 98 96
query43 510 492 470 470
query44 1135 755 757 755
query45 194 169 161 161
query46 1085 757 730 730
query47 1876 1786 1788 1786
query48 370 295 300 295
query49 767 431 439 431
query50 809 411 427 411
query51 6990 6893 6943 6893
query52 103 88 84 84
query53 249 177 177 177
query54 561 454 453 453
query55 77 76 74 74
query56 264 270 257 257
query57 1213 1088 1061 1061
query58 222 244 244 244
query59 3041 3008 3026 3008
query60 299 274 266 266
query61 104 101 100 100
query62 765 651 684 651
query63 214 188 195 188
query64 1372 689 669 669
query65 3282 3174 3185 3174
query66 657 342 338 338
query67 15651 15319 15178 15178
query68 1980 853 851 851
query69 434 327 316 316
query70 1090 1193 1166 1166
query71 346 344 335 335
query72 4626 3561 3540 3540
query73 588 567 580 567
query74 9188 8856 8855 8855
query75 3023 2969 2939 2939
query76 944 843 836 836
query77 510 399 401 399
query78 9375 9275 9305 9275
query79 896 869 843 843
query80 778 801 787 787
query81 456 258 266 258
query82 263 259 268 259
query83 198 188 186 186
query84 201 107 104 104
query85 624 391 427 391
query86 324 314 313 313
query87 4343 4354 4331 4331
query88 4224 4091 4080 4080
query89 368 360 357 357
query90 1418 313 305 305
query91 120 124 120 120
query92 73 75 80 75
query93 907 907 903 903
query94 519 361 349 349
query95 413 408 410 408
query96 462 467 468 467
query97 3097 3107 3120 3107
query98 228 226 218 218
query99 1302 1270 1305 1270
Total cold run time: 295778 ms
Total hot run time: 198262 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2338131205
<details>
<summary>TPC-H: <b>Total hot run time: 37815 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit adb4e0f84651db6193b6cbc442c680f0c8d222a8, data reload: false
------ Round 1 ----------------------------------
q1 17629 4650 4293 4293
q2 2021 193 181 181
q3 11884 951 1239 951
q4 10511 687 796 687
q5 7756 2860 2866 2860
q6 237 143 141 141
q7 959 613 603 603
q8 9545 2101 2116 2101
q9 8011 6574 6571 6571
q10 7010 2175 2238 2175
q11 456 245 253 245
q12 402 226 225 225
q13 18899 3112 3079 3079
q14 275 228 231 228
q15 541 501 486 486
q16 534 455 437 437
q17 993 684 671 671
q18 7403 6909 6842 6842
q19 1386 1141 975 975
q20 662 338 331 331
q21 4001 2722 2929 2722
q22 1104 1022 1011 1011
Total cold run time: 112219 ms
Total hot run time: 37815 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4381 4317 4267 4267
q2 381 289 260 260
q3 2879 2701 2678 2678
q4 1977 1659 1706 1659
q5 5702 5693 5769 5693
q6 227 132 132 132
q7 2244 1815 1863 1815
q8 3293 3502 3499 3499
q9 8881 8868 8813 8813
q10 3681 3378 3312 3312
q11 589 518 511 511
q12 834 637 666 637
q13 13149 3298 3310 3298
q14 321 288 301 288
q15 538 491 493 491
q16 562 521 504 504
q17 1840 1576 1531 1531
q18 8188 7727 7882 7727
q19 1732 1670 1565 1565
q20 2196 1887 1966 1887
q21 5764 5471 5448 5448
q22 1125 1050 1007 1007
Total cold run time: 70484 ms
Total hot run time: 57022 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2340003078
TeamCity be ut coverage result:
Function Coverage: 36.82% (9394/25510)
Line Coverage: 28.21% (77461/274558)
Region Coverage: 27.63% (40006/144815)
Branch Coverage: 24.25% (20345/83906)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9792c23dbcd67c71efde6e320dc0cf35ea52e065_9792c23dbcd67c71efde6e320dc0cf35ea52e065/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40559:
URL: https://github.com/apache/doris/pull/40559#issuecomment-2342678301
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](join)Support MethodOneString to optimize hash join with a single string key [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #40559:
URL: https://github.com/apache/doris/pull/40559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org