You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zzzxl1993 (via GitHub)" <gi...@apache.org> on 2024/07/12 10:28:16 UTC
[PR] [feature](inverted index) multi_match function add [doris]
zzzxl1993 opened a new pull request, #37722:
URL: https://github.com/apache/doris/pull/37722
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2228770706
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239394604
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239286348
<details>
<summary>TPC-DS: <b>Total hot run time: 172760 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false
query1 917 395 376 376
query2 6445 1857 1753 1753
query3 6631 211 217 211
query4 28153 17427 17282 17282
query5 3675 466 475 466
query6 254 179 159 159
query7 4548 283 290 283
query8 237 195 189 189
query9 8441 2404 2380 2380
query10 418 271 282 271
query11 11199 10043 10083 10043
query12 114 83 85 83
query13 1589 357 363 357
query14 10110 7431 7654 7431
query15 211 162 160 160
query16 7592 475 450 450
query17 1117 558 517 517
query18 1951 280 274 274
query19 186 147 146 146
query20 86 83 76 76
query21 204 130 127 127
query22 4310 4146 3991 3991
query23 34003 33777 33578 33578
query24 10968 2984 2917 2917
query25 575 393 397 393
query26 717 146 151 146
query27 2289 280 277 277
query28 6041 2049 2049 2049
query29 888 619 632 619
query30 258 148 149 148
query31 1016 749 754 749
query32 104 54 59 54
query33 662 368 330 330
query34 911 494 501 494
query35 884 742 747 742
query36 1147 992 973 973
query37 140 77 89 77
query38 3018 2872 2821 2821
query39 935 837 879 837
query40 214 118 122 118
query41 47 44 42 42
query42 113 99 97 97
query43 490 448 449 448
query44 1133 726 718 718
query45 194 165 160 160
query46 1093 713 743 713
query47 1868 1771 1784 1771
query48 359 295 298 295
query49 844 417 422 417
query50 789 396 403 396
query51 6927 6766 6787 6766
query52 112 88 97 88
query53 351 286 287 286
query54 881 453 456 453
query55 77 76 73 73
query56 307 280 292 280
query57 1124 1049 1054 1049
query58 256 265 255 255
query59 2835 2679 2484 2484
query60 313 319 292 292
query61 113 116 113 113
query62 815 638 667 638
query63 312 292 285 285
query64 9621 2325 1656 1656
query65 3153 3119 3139 3119
query66 755 336 321 321
query67 15359 14900 15103 14900
query68 4561 533 542 533
query69 615 392 352 352
query70 1208 1120 1124 1120
query71 439 283 275 275
query72 8791 5080 5859 5080
query73 756 323 346 323
query74 6073 5697 5642 5642
query75 4024 2700 2664 2664
query76 3013 972 816 816
query77 690 316 306 306
query78 9725 9166 8945 8945
query79 2910 538 515 515
query80 1399 471 479 471
query81 600 219 222 219
query82 947 139 131 131
query83 359 164 166 164
query84 269 91 86 86
query85 1780 311 299 299
query86 459 332 325 325
query87 3251 3107 3097 3097
query88 4208 2367 2384 2367
query89 488 382 382 382
query90 1874 187 196 187
query91 129 98 97 97
query92 60 51 49 49
query93 3747 498 501 498
query94 1259 284 287 284
query95 414 385 320 320
query96 605 272 285 272
query97 3176 3033 3020 3020
query98 226 203 200 200
query99 1479 1259 1261 1259
Total cold run time: 282933 ms
Total hot run time: 172760 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242174476
<details>
<summary>TPC-DS: <b>Total hot run time: 174180 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false
query1 920 380 378 378
query2 6451 1918 1894 1894
query3 6653 203 219 203
query4 23729 17685 17391 17391
query5 4204 495 506 495
query6 276 202 194 194
query7 4601 296 290 290
query8 249 199 202 199
query9 8369 2450 2419 2419
query10 443 275 272 272
query11 10634 10201 9994 9994
query12 131 84 86 84
query13 1631 366 385 366
query14 9062 7959 7492 7492
query15 209 170 165 165
query16 7584 495 517 495
query17 1321 532 528 528
query18 1768 277 283 277
query19 205 147 150 147
query20 94 84 84 84
query21 212 138 132 132
query22 4276 4147 4073 4073
query23 33838 34665 33372 33372
query24 12142 2972 2848 2848
query25 529 370 369 369
query26 1235 151 151 151
query27 2977 270 276 270
query28 7376 2016 2021 2016
query29 875 625 631 625
query30 283 149 147 147
query31 941 744 739 739
query32 96 53 55 53
query33 793 388 337 337
query34 900 490 498 490
query35 858 734 731 731
query36 1110 919 909 909
query37 218 80 80 80
query38 2840 2745 2753 2745
query39 894 814 825 814
query40 281 119 124 119
query41 47 46 44 44
query42 117 100 100 100
query43 507 476 475 475
query44 1210 735 738 735
query45 199 165 164 164
query46 1094 747 730 730
query47 1869 1788 1790 1788
query48 378 297 286 286
query49 1168 405 434 405
query50 783 413 411 411
query51 6899 6698 6735 6698
query52 110 90 97 90
query53 359 287 294 287
query54 898 459 454 454
query55 75 77 78 77
query56 292 269 266 266
query57 1141 1047 1048 1047
query58 256 249 249 249
query59 2962 2574 2699 2574
query60 298 278 289 278
query61 104 100 96 96
query62 840 642 650 642
query63 354 300 293 293
query64 10423 2197 1687 1687
query65 3165 3342 3168 3168
query66 1269 363 325 325
query67 15386 14936 15018 14936
query68 4562 533 552 533
query69 468 359 354 354
query70 1137 1147 1089 1089
query71 448 283 288 283
query72 7139 5675 5731 5675
query73 757 337 333 333
query74 6176 5670 5686 5670
query75 3381 2698 2671 2671
query76 2819 1037 942 942
query77 461 326 332 326
query78 12375 10558 9050 9050
query79 6576 532 545 532
query80 1223 526 514 514
query81 582 225 238 225
query82 452 146 137 137
query83 275 180 173 173
query84 283 90 96 90
query85 778 380 350 350
query86 498 312 325 312
query87 3303 3141 3139 3139
query88 4092 2451 2460 2451
query89 472 387 393 387
query90 1940 211 201 201
query91 140 116 114 114
query92 69 55 52 52
query93 2707 514 507 507
query94 1132 309 313 309
query95 436 338 339 338
query96 613 417 279 279
query97 3241 3028 3051 3028
query98 226 197 196 196
query99 1727 1315 1228 1228
Total cold run time: 283899 ms
Total hot run time: 174180 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2230751147
<details>
<summary>TPC-DS: <b>Total hot run time: 174442 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false
query1 914 388 370 370
query2 6419 1869 1875 1869
query3 6631 205 225 205
query4 28116 17566 17249 17249
query5 3767 485 480 480
query6 288 156 176 156
query7 4579 303 285 285
query8 235 185 187 185
query9 8563 2472 2427 2427
query10 454 285 272 272
query11 12433 10013 10168 10013
query12 113 86 82 82
query13 1641 365 361 361
query14 10158 7683 7634 7634
query15 216 165 166 165
query16 7719 315 321 315
query17 1753 565 545 545
query18 1966 291 280 280
query19 195 150 150 150
query20 91 80 83 80
query21 205 131 131 131
query22 4206 4204 4062 4062
query23 33879 33774 33699 33699
query24 10701 2936 2951 2936
query25 579 389 396 389
query26 706 152 160 152
query27 2329 281 284 281
query28 6086 2134 2090 2090
query29 896 648 652 648
query30 249 151 150 150
query31 976 765 755 755
query32 91 58 56 56
query33 725 323 327 323
query34 880 519 521 519
query35 683 627 588 588
query36 1135 999 981 981
query37 148 86 85 85
query38 2971 2871 2848 2848
query39 893 875 841 841
query40 202 118 115 115
query41 49 40 40 40
query42 116 98 103 98
query43 503 469 480 469
query44 1089 730 754 730
query45 192 159 160 159
query46 1082 747 736 736
query47 1943 1799 1808 1799
query48 374 301 309 301
query49 858 451 410 410
query50 791 386 389 386
query51 6863 6812 6793 6793
query52 102 90 94 90
query53 352 289 287 287
query54 906 454 451 451
query55 75 76 74 74
query56 290 265 275 265
query57 1104 1079 1048 1048
query58 255 248 252 248
query59 2828 2590 2570 2570
query60 306 269 276 269
query61 97 97 113 97
query62 783 660 632 632
query63 322 291 284 284
query64 9183 2296 1673 1673
query65 3182 3150 3135 3135
query66 748 329 331 329
query67 15523 15237 14936 14936
query68 4503 562 552 552
query69 564 403 346 346
query70 1132 1124 1155 1124
query71 372 279 278 278
query72 7029 5695 5866 5695
query73 764 331 326 326
query74 6188 5749 5786 5749
query75 3343 2676 2716 2676
query76 2122 924 952 924
query77 431 302 304 302
query78 9866 9202 9190 9190
query79 2119 535 525 525
query80 1335 486 458 458
query81 602 222 221 221
query82 667 132 136 132
query83 243 167 166 166
query84 249 90 85 85
query85 1739 310 304 304
query86 474 333 303 303
query87 3308 3137 3093 3093
query88 4035 2397 2385 2385
query89 465 386 386 386
query90 1733 190 192 190
query91 126 99 104 99
query92 63 48 49 48
query93 2196 515 518 515
query94 1051 218 212 212
query95 406 322 319 319
query96 587 283 271 271
query97 3213 3040 3078 3040
query98 211 201 195 195
query99 1533 1252 1282 1252
Total cold run time: 277376 ms
Total hot run time: 174442 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #37722:
URL: https://github.com/apache/doris/pull/37722#discussion_r1684508975
##########
be/src/vec/functions/function_multi_match.cpp:
##########
@@ -0,0 +1,189 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/functions/function_multi_match.h"
+
+#include <gen_cpp/PaloBrokerService_types.h>
+#include <glog/logging.h>
+
+#include <boost/algorithm/string.hpp>
+#include <boost/algorithm/string/classification.hpp>
+#include <cstddef>
+#include <cstdint>
+#include <memory>
+#include <roaring/roaring.hh>
+#include <string>
+#include <vector>
+
+#include "io/fs/file_reader.h"
+#include "olap/olap_common.h"
+#include "olap/rowset/segment_v2/inverted_index/query/phrase_prefix_query.h"
+#include "olap/rowset/segment_v2/segment_iterator.h"
+#include "runtime/primitive_type.h"
+#include "vec/columns/column.h"
+#include "vec/data_types/data_type.h"
+#include "vec/exprs/varray_literal.h"
+#include "vec/exprs/vexpr.h"
+#include "vec/exprs/vslot_ref.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+Status FunctionMultiMatch::execute_impl(FunctionContext* /*context*/, Block& block,
+ const ColumnNumbers& arguments, size_t result,
+ size_t /*input_rows_count*/) const {
+ return Status::RuntimeError("only inverted index queries are supported");
+}
+
+Status FunctionMultiMatch::open(FunctionContext* context,
+ FunctionContext::FunctionStateScope scope) {
+ if (scope == FunctionContext::THREAD_LOCAL) {
+ return Status::OK();
+ }
+
+ DCHECK(context->get_num_args() == 4);
+ for (int i = 0; i < context->get_num_args(); ++i) {
+ DCHECK(is_string_type(context->get_arg_type(i)->type));
+ }
+
+ std::shared_ptr<MatchParam> state = std::make_shared<MatchParam>();
+ context->set_function_state(scope, state);
+ for (int i = 0; i < context->get_num_args(); ++i) {
+ const auto& const_column_ptr = context->get_constant_col(i);
+ if (const_column_ptr) {
+ auto const_data = const_column_ptr->column_ptr->get_data_at(0);
+ switch (i) {
+ case 1: {
+ std::string field_names_str = const_data.to_string();
+ field_names_str.erase(
+ std::remove_if(field_names_str.begin(), field_names_str.end(),
+ [](unsigned char c) { return std::isspace(c); }),
+ field_names_str.end());
+ std::vector<std::string> field_names;
+ boost::split(field_names, field_names_str, boost::algorithm::is_any_of(","));
+ state->fields.insert(field_names.begin(), field_names.end());
+ } break;
+ case 2:
+ state->type = const_data.to_string();
+ break;
+ case 3:
+ state->query = const_data.to_string();
+ break;
+ default:
+ break;
+ }
+ }
+ }
+
+ return Status::OK();
+}
+
+Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
Review Comment:
warning: function 'eval_inverted_index' exceeds recommended size/complexity thresholds [readability-function-size]
```cpp
Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
^
```
<details>
<summary>Additional context</summary>
**be/src/vec/functions/function_multi_match.cpp:94:** 86 lines including whitespace and comments (threshold 80)
```cpp
Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2229969561
<details>
<summary>ClickBench: <b>Total hot run time: 31.17 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.22 0.06 0.05
query4 1.66 0.08 0.08
query5 0.50 0.49 0.49
query6 1.14 0.73 0.73
query7 0.02 0.01 0.01
query8 0.04 0.04 0.04
query9 0.53 0.50 0.49
query10 0.55 0.54 0.54
query11 0.16 0.11 0.11
query12 0.14 0.12 0.12
query13 0.59 0.59 0.59
query14 0.77 0.79 0.77
query15 0.87 0.82 0.82
query16 0.34 0.37 0.36
query17 0.97 1.03 0.96
query18 0.22 0.22 0.21
query19 1.74 1.70 1.82
query20 0.01 0.01 0.01
query21 15.41 0.76 0.67
query22 4.22 7.10 2.45
query23 18.31 1.34 1.26
query24 2.09 0.24 0.22
query25 0.15 0.09 0.08
query26 0.29 0.21 0.20
query27 0.46 0.23 0.23
query28 13.26 1.01 1.00
query29 12.60 3.30 3.32
query30 0.25 0.06 0.05
query31 2.86 0.39 0.39
query32 3.29 0.48 0.47
query33 2.94 2.90 2.92
query34 16.95 4.32 4.39
query35 4.47 4.44 4.41
query36 0.65 0.49 0.51
query37 0.18 0.16 0.16
query38 0.15 0.14 0.15
query39 0.05 0.03 0.03
query40 0.15 0.13 0.11
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 109.52 s
Total hot run time: 31.17 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239547035
<details>
<summary>ClickBench: <b>Total hot run time: 30.54 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.06
query4 1.66 0.08 0.09
query5 0.49 0.50 0.50
query6 1.12 0.72 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.49 0.50
query10 0.53 0.55 0.55
query11 0.16 0.11 0.11
query12 0.15 0.13 0.12
query13 0.61 0.58 0.58
query14 0.76 0.77 0.78
query15 0.86 0.82 0.82
query16 0.36 0.37 0.35
query17 1.03 1.04 1.00
query18 0.22 0.22 0.21
query19 1.92 1.74 1.84
query20 0.01 0.00 0.01
query21 15.42 0.80 0.66
query22 3.78 7.85 1.96
query23 18.25 1.33 1.24
query24 2.13 0.23 0.24
query25 0.16 0.09 0.09
query26 0.29 0.21 0.20
query27 0.46 0.23 0.22
query28 13.20 1.02 1.00
query29 12.61 3.34 3.26
query30 0.25 0.06 0.05
query31 2.88 0.39 0.38
query32 3.27 0.49 0.48
query33 2.91 2.83 2.94
query34 16.90 4.30 4.34
query35 4.41 4.44 4.39
query36 0.66 0.48 0.48
query37 0.19 0.15 0.16
query38 0.16 0.14 0.14
query39 0.05 0.03 0.04
query40 0.16 0.13 0.12
query41 0.09 0.04 0.05
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.18 s
Total hot run time: 30.54 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242101231
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242181847
<details>
<summary>ClickBench: <b>Total hot run time: 31.13 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.06 0.05
query4 1.68 0.08 0.08
query5 0.50 0.49 0.48
query6 1.12 0.74 0.74
query7 0.02 0.02 0.01
query8 0.05 0.05 0.05
query9 0.54 0.48 0.49
query10 0.54 0.56 0.53
query11 0.16 0.12 0.12
query12 0.14 0.12 0.12
query13 0.60 0.59 0.59
query14 0.76 0.78 0.79
query15 0.85 0.82 0.82
query16 0.37 0.36 0.37
query17 1.03 1.04 0.97
query18 0.24 0.23 0.23
query19 1.92 1.74 1.83
query20 0.01 0.01 0.01
query21 15.42 0.77 0.66
query22 4.50 6.90 2.21
query23 18.33 1.43 1.27
query24 2.11 0.23 0.23
query25 0.17 0.09 0.08
query26 0.29 0.22 0.21
query27 0.45 0.23 0.24
query28 13.32 1.02 0.99
query29 12.62 3.33 3.30
query30 0.25 0.06 0.06
query31 2.89 0.38 0.38
query32 3.30 0.49 0.48
query33 2.95 2.89 2.93
query34 16.76 4.43 4.42
query35 4.48 4.41 4.47
query36 0.65 0.49 0.49
query37 0.19 0.16 0.15
query38 0.16 0.14 0.15
query39 0.04 0.03 0.04
query40 0.16 0.12 0.12
query41 0.10 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.12 s
Total hot run time: 31.13 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2230612385
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239067288
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2230730602
<details>
<summary>TPC-H: <b>Total hot run time: 40212 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false
------ Round 1 ----------------------------------
q1 17595 4394 4381 4381
q2 2009 192 193 192
q3 10444 1269 1073 1073
q4 10190 793 813 793
q5 7591 2719 2728 2719
q6 217 135 138 135
q7 970 607 609 607
q8 9221 2099 2123 2099
q9 8746 6610 6624 6610
q10 8837 3860 3778 3778
q11 482 242 246 242
q12 429 236 236 236
q13 17765 3027 2996 2996
q14 276 231 232 231
q15 533 495 497 495
q16 509 393 393 393
q17 985 664 621 621
q18 8103 7497 7441 7441
q19 6770 1407 1355 1355
q20 702 316 317 316
q21 5088 3219 3242 3219
q22 354 280 294 280
Total cold run time: 117816 ms
Total hot run time: 40212 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4383 4273 4248 4248
q2 388 267 277 267
q3 3046 2959 2941 2941
q4 2112 1716 1767 1716
q5 5616 5619 5518 5518
q6 220 137 139 137
q7 2309 1828 1841 1828
q8 3307 3447 3438 3438
q9 8816 8917 8834 8834
q10 4139 3829 3944 3829
q11 601 503 491 491
q12 844 650 631 631
q13 16051 3169 3213 3169
q14 337 281 288 281
q15 531 488 498 488
q16 496 434 434 434
q17 1821 1523 1488 1488
q18 8137 8029 7849 7849
q19 1793 1660 1479 1479
q20 2079 1885 1871 1871
q21 5237 4728 4885 4728
q22 597 514 542 514
Total cold run time: 72860 ms
Total hot run time: 56179 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242462812
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye merged PR #37722:
URL: https://github.com/apache/doris/pull/37722
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2225286221
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239263223
<details>
<summary>TPC-H: <b>Total hot run time: 39945 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false
------ Round 1 ----------------------------------
q1 17611 4625 4336 4336
q2 2020 190 183 183
q3 10448 1272 1078 1078
q4 10197 778 689 689
q5 7561 2686 2687 2686
q6 223 136 138 136
q7 958 602 598 598
q8 9223 2089 2101 2089
q9 8583 6571 6544 6544
q10 8827 3752 3819 3752
q11 478 234 232 232
q12 403 224 224 224
q13 18844 2968 2957 2957
q14 275 229 228 228
q15 540 482 504 482
q16 506 385 371 371
q17 973 655 722 655
q18 8192 7529 7439 7439
q19 6125 1425 1411 1411
q20 661 313 323 313
q21 5246 3326 3269 3269
q22 339 273 288 273
Total cold run time: 118233 ms
Total hot run time: 39945 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4425 4250 4300 4250
q2 383 267 258 258
q3 3003 2883 2933 2883
q4 1991 1696 1747 1696
q5 5627 5510 5484 5484
q6 220 137 136 136
q7 2186 1876 1840 1840
q8 3277 3399 3456 3399
q9 8786 8812 8842 8812
q10 4058 3813 3750 3750
q11 596 495 500 495
q12 832 626 632 626
q13 15972 3189 3157 3157
q14 338 279 293 279
q15 521 492 481 481
q16 487 451 435 435
q17 1851 1524 1523 1523
q18 8139 8018 7900 7900
q19 1805 1691 1505 1505
q20 2822 1873 1844 1844
q21 9319 5061 4703 4703
q22 583 536 479 479
Total cold run time: 77221 ms
Total hot run time: 55935 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239298886
<details>
<summary>ClickBench: <b>Total hot run time: 30.9 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b45cfd31514f26f29acadea60c06706c5a25f0f0, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.03
query3 0.22 0.05 0.05
query4 1.68 0.07 0.07
query5 0.54 0.48 0.49
query6 1.13 0.73 0.72
query7 0.01 0.01 0.02
query8 0.05 0.04 0.04
query9 0.55 0.49 0.48
query10 0.53 0.53 0.55
query11 0.15 0.11 0.11
query12 0.14 0.12 0.13
query13 0.60 0.60 0.58
query14 0.76 0.78 0.77
query15 0.85 0.80 0.82
query16 0.36 0.37 0.35
query17 1.03 0.99 0.99
query18 0.24 0.23 0.22
query19 1.93 1.81 1.69
query20 0.01 0.00 0.01
query21 15.39 0.79 0.65
query22 4.04 6.53 2.18
query23 18.29 1.37 1.35
query24 2.07 0.23 0.24
query25 0.16 0.09 0.08
query26 0.31 0.21 0.21
query27 0.45 0.24 0.23
query28 13.26 1.02 1.00
query29 12.63 3.41 3.36
query30 0.25 0.06 0.06
query31 2.88 0.39 0.38
query32 3.26 0.48 0.46
query33 2.91 2.92 2.86
query34 17.08 4.36 4.40
query35 4.39 4.39 4.46
query36 0.64 0.46 0.46
query37 0.18 0.15 0.16
query38 0.16 0.15 0.15
query39 0.05 0.04 0.03
query40 0.15 0.12 0.13
query41 0.09 0.05 0.06
query42 0.05 0.04 0.04
query43 0.04 0.04 0.03
Total cold run time: 109.63 s
Total hot run time: 30.9 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2230761676
<details>
<summary>ClickBench: <b>Total hot run time: 30.71 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed9f6e73cce3e1b687a25afeeb7cacdb53b02be0, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.07 0.08
query5 0.50 0.49 0.49
query6 1.14 0.73 0.73
query7 0.02 0.02 0.01
query8 0.04 0.04 0.04
query9 0.55 0.51 0.50
query10 0.57 0.55 0.55
query11 0.14 0.12 0.12
query12 0.14 0.12 0.12
query13 0.58 0.59 0.58
query14 0.77 0.76 0.79
query15 0.85 0.81 0.81
query16 0.36 0.36 0.36
query17 0.99 0.96 0.96
query18 0.23 0.22 0.22
query19 1.81 1.70 1.70
query20 0.01 0.01 0.01
query21 15.39 0.74 0.65
query22 4.11 7.12 1.99
query23 18.24 1.51 1.30
query24 2.08 0.26 0.21
query25 0.15 0.09 0.08
query26 0.28 0.20 0.22
query27 0.45 0.24 0.22
query28 13.26 1.03 1.00
query29 12.57 3.28 3.29
query30 0.25 0.06 0.07
query31 2.84 0.40 0.39
query32 3.26 0.48 0.47
query33 2.88 2.86 2.94
query34 17.19 4.46 4.41
query35 4.45 4.42 4.42
query36 0.66 0.49 0.47
query37 0.20 0.15 0.16
query38 0.15 0.15 0.14
query39 0.05 0.04 0.03
query40 0.15 0.12 0.11
query41 0.10 0.05 0.04
query42 0.06 0.05 0.05
query43 0.05 0.04 0.03
Total cold run time: 109.54 s
Total hot run time: 30.71 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239539454
<details>
<summary>TPC-DS: <b>Total hot run time: 173229 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false
query1 900 373 371 371
query2 6427 1941 1841 1841
query3 6655 214 224 214
query4 28361 17275 17552 17275
query5 3704 467 475 467
query6 276 169 166 166
query7 4582 288 285 285
query8 244 202 192 192
query9 8554 2497 2447 2447
query10 430 295 280 280
query11 11141 9984 10092 9984
query12 113 88 84 84
query13 1640 372 371 371
query14 10325 7211 7658 7211
query15 213 167 170 167
query16 7677 493 468 468
query17 1571 572 541 541
query18 1928 285 278 278
query19 202 155 157 155
query20 93 87 85 85
query21 207 126 127 126
query22 4208 4090 4035 4035
query23 34141 33841 33508 33508
query24 11085 2972 2909 2909
query25 640 415 411 411
query26 1157 155 158 155
query27 2866 285 294 285
query28 7413 2110 2102 2102
query29 1002 659 638 638
query30 264 159 155 155
query31 1004 744 724 724
query32 103 52 55 52
query33 780 345 330 330
query34 976 491 501 491
query35 868 759 726 726
query36 1109 983 959 959
query37 156 82 81 81
query38 2969 2898 2838 2838
query39 912 849 845 845
query40 199 120 114 114
query41 47 43 43 43
query42 118 99 101 99
query43 515 470 468 468
query44 1185 744 727 727
query45 192 159 162 159
query46 1089 745 746 745
query47 1847 1740 1758 1740
query48 361 294 287 287
query49 836 411 434 411
query50 776 402 392 392
query51 6900 6794 6770 6770
query52 100 94 91 91
query53 352 290 298 290
query54 853 451 448 448
query55 77 73 75 73
query56 308 263 266 263
query57 1128 1059 1038 1038
query58 242 250 261 250
query59 2944 2570 2537 2537
query60 306 271 273 271
query61 93 98 92 92
query62 802 661 653 653
query63 313 296 288 288
query64 9460 2224 1662 1662
query65 3157 3078 3079 3078
query66 770 315 327 315
query67 15391 14850 14835 14835
query68 5340 552 544 544
query69 718 454 359 359
query70 1217 1112 1146 1112
query71 486 280 279 279
query72 9234 5700 5525 5525
query73 784 324 326 324
query74 6083 5670 5623 5623
query75 4300 2690 2641 2641
query76 4007 989 908 908
query77 674 319 311 311
query78 9727 8999 10368 8999
query79 3763 533 530 530
query80 2294 480 482 480
query81 604 225 219 219
query82 1322 138 132 132
query83 309 163 164 163
query84 278 93 87 87
query85 1579 315 301 301
query86 462 303 312 303
query87 3281 3128 3116 3116
query88 4070 2395 2428 2395
query89 469 372 382 372
query90 1913 194 192 192
query91 135 100 99 99
query92 62 49 50 49
query93 2807 503 517 503
query94 1300 291 266 266
query95 404 308 310 308
query96 602 271 278 271
query97 3252 2994 3019 2994
query98 217 199 191 191
query99 1496 1288 1242 1242
Total cold run time: 290446 ms
Total hot run time: 173229 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2239522547
<details>
<summary>TPC-H: <b>Total hot run time: 39971 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a926b4ca357dc3e85c4fda809f47acad3842f0e2, data reload: false
------ Round 1 ----------------------------------
q1 17625 4439 4312 4312
q2 2010 191 184 184
q3 10472 1249 1168 1168
q4 10184 777 753 753
q5 7561 2713 2708 2708
q6 227 139 134 134
q7 962 595 587 587
q8 9218 2072 2118 2072
q9 8698 6556 6599 6556
q10 8846 3768 3795 3768
q11 450 235 233 233
q12 434 221 215 215
q13 17777 2985 2969 2969
q14 280 235 238 235
q15 525 490 499 490
q16 499 389 376 376
q17 969 709 737 709
q18 8233 7504 7430 7430
q19 6819 1458 1345 1345
q20 684 323 332 323
q21 4903 3121 3992 3121
q22 344 290 283 283
Total cold run time: 117720 ms
Total hot run time: 39971 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4406 4230 4261 4230
q2 373 274 273 273
q3 3002 2944 2913 2913
q4 1952 1677 1708 1677
q5 5659 5562 5486 5486
q6 222 128 129 128
q7 2188 1862 1854 1854
q8 3280 3426 3489 3426
q9 8790 8865 8850 8850
q10 4076 3807 3838 3807
q11 596 521 512 512
q12 808 622 612 612
q13 17146 3124 3172 3124
q14 325 298 279 279
q15 545 504 483 483
q16 498 443 431 431
q17 1823 1552 1529 1529
q18 8321 8015 7871 7871
q19 1861 1743 1619 1619
q20 2096 1903 1860 1860
q21 9202 4707 4748 4707
q22 581 511 495 495
Total cold run time: 77750 ms
Total hot run time: 56166 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2229884784
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242462722
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2229961579
<details>
<summary>TPC-DS: <b>Total hot run time: 174312 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false
query1 915 372 371 371
query2 6455 1859 1846 1846
query3 6634 204 216 204
query4 23521 17355 17143 17143
query5 3770 481 475 475
query6 267 164 162 162
query7 4588 295 282 282
query8 244 187 188 187
query9 8399 2469 2479 2469
query10 443 293 279 279
query11 11072 10148 10150 10148
query12 114 99 84 84
query13 1648 391 364 364
query14 10094 8026 7806 7806
query15 238 165 167 165
query16 7692 327 323 323
query17 1788 567 544 544
query18 1310 294 285 285
query19 202 158 160 158
query20 91 86 82 82
query21 204 129 127 127
query22 4297 4119 4145 4119
query23 34010 33724 33758 33724
query24 10827 2917 3085 2917
query25 598 406 384 384
query26 704 150 148 148
query27 2239 285 279 279
query28 6031 2098 2089 2089
query29 884 646 622 622
query30 257 154 155 154
query31 958 760 770 760
query32 96 52 56 52
query33 705 295 301 295
query34 897 490 488 488
query35 692 586 598 586
query36 1117 985 976 976
query37 157 87 86 86
query38 3006 2929 2853 2853
query39 911 863 866 863
query40 215 124 129 124
query41 54 45 44 44
query42 116 103 102 102
query43 506 451 467 451
query44 1106 712 730 712
query45 197 159 159 159
query46 1077 723 750 723
query47 1861 1794 1781 1781
query48 389 292 287 287
query49 838 401 416 401
query50 772 396 395 395
query51 6866 6733 6743 6733
query52 110 99 94 94
query53 356 281 288 281
query54 902 451 446 446
query55 72 72 73 72
query56 303 263 262 262
query57 1129 1058 1061 1058
query58 237 241 246 241
query59 2829 2613 2641 2613
query60 293 280 273 273
query61 94 94 120 94
query62 790 649 640 640
query63 314 285 283 283
query64 9096 2214 1680 1680
query65 3164 3115 3120 3115
query66 761 325 320 320
query67 15504 14954 15065 14954
query68 4986 537 540 537
query69 694 484 335 335
query70 1182 1122 1129 1122
query71 485 281 275 275
query72 9046 5656 6128 5656
query73 771 327 325 325
query74 6228 5638 5689 5638
query75 4429 2699 2735 2699
query76 3612 959 924 924
query77 748 323 308 308
query78 11964 9333 9035 9035
query79 5855 517 519 517
query80 936 483 474 474
query81 583 225 218 218
query82 675 133 135 133
query83 377 166 171 166
query84 281 86 89 86
query85 704 383 291 291
query86 431 321 321 321
query87 3388 3115 3103 3103
query88 4708 2447 2472 2447
query89 480 375 375 375
query90 1961 198 193 193
query91 129 100 103 100
query92 62 49 58 49
query93 3773 504 495 495
query94 1218 212 218 212
query95 404 316 314 314
query96 607 290 276 276
query97 3206 3023 3049 3023
query98 226 204 196 196
query99 1556 1232 1233 1232
Total cold run time: 283731 ms
Total hot run time: 174312 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2229953010
<details>
<summary>TPC-H: <b>Total hot run time: 39750 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 718b16619cf923341f31db2e2a8f3b28ae1f1052, data reload: false
------ Round 1 ----------------------------------
q1 17607 4949 4288 4288
q2 2022 194 184 184
q3 10540 1232 1141 1141
q4 10202 842 781 781
q5 7549 2789 2613 2613
q6 217 134 136 134
q7 961 594 603 594
q8 9341 2090 2081 2081
q9 8806 6616 6596 6596
q10 8711 3843 3783 3783
q11 437 231 238 231
q12 410 222 217 217
q13 17771 2954 2965 2954
q14 275 245 250 245
q15 516 494 488 488
q16 513 404 376 376
q17 981 659 656 656
q18 8046 7514 7333 7333
q19 4903 1319 1465 1319
q20 699 333 315 315
q21 4916 3151 3134 3134
q22 348 301 287 287
Total cold run time: 115771 ms
Total hot run time: 39750 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4395 4214 4280 4214
q2 382 274 273 273
q3 3015 2863 2851 2851
q4 2005 1723 1678 1678
q5 5710 5522 5556 5522
q6 224 137 131 131
q7 2219 1893 1870 1870
q8 3300 3428 3440 3428
q9 8826 8788 8939 8788
q10 4072 3878 3809 3809
q11 605 518 510 510
q12 828 629 641 629
q13 17303 3139 3209 3139
q14 331 286 299 286
q15 535 496 494 494
q16 500 438 466 438
q17 1817 1497 1506 1497
q18 8301 7804 7833 7804
q19 2261 1513 1569 1513
q20 2182 1899 1842 1842
q21 10280 4641 4740 4641
q22 603 511 509 509
Total cold run time: 79694 ms
Total hot run time: 55866 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #37722:
URL: https://github.com/apache/doris/pull/37722#discussion_r1678020744
##########
be/src/vec/functions/function_multi_match.cpp:
##########
@@ -0,0 +1,187 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/functions/function_multi_match.h"
+
+#include <gen_cpp/PaloBrokerService_types.h>
+#include <glog/logging.h>
+
+#include <boost/algorithm/string.hpp>
+#include <boost/algorithm/string/classification.hpp>
+#include <cstddef>
+#include <cstdint>
+#include <memory>
+#include <roaring/roaring.hh>
+#include <string>
+#include <vector>
+
+#include "io/fs/file_reader.h"
+#include "olap/olap_common.h"
+#include "olap/rowset/segment_v2/inverted_index/query/phrase_prefix_query.h"
+#include "olap/rowset/segment_v2/segment_iterator.h"
+#include "runtime/primitive_type.h"
+#include "vec/columns/column.h"
+#include "vec/data_types/data_type.h"
+#include "vec/exprs/varray_literal.h"
+#include "vec/exprs/vexpr.h"
+#include "vec/exprs/vslot_ref.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+Status FunctionMultiMatch::execute_impl(FunctionContext* /*context*/, Block& block,
+ const ColumnNumbers& arguments, size_t result,
+ size_t /*input_rows_count*/) const {
+ return Status::RuntimeError("only inverted index queries are supported");
+}
+
+Status FunctionMultiMatch::open(FunctionContext* context,
+ FunctionContext::FunctionStateScope scope) {
+ if (scope == FunctionContext::THREAD_LOCAL) {
+ return Status::OK();
+ }
+
+ DCHECK(context->get_num_args() == 4);
+ for (int i = 0; i < context->get_num_args(); ++i) {
+ DCHECK(is_string_type(context->get_arg_type(i)->type));
+ }
+
+ std::shared_ptr<MatchParam> state = std::make_shared<MatchParam>();
+ context->set_function_state(scope, state);
+ for (int i = 0; i < context->get_num_args(); ++i) {
+ const auto& const_column_ptr = context->get_constant_col(i);
+ if (const_column_ptr) {
+ auto const_data = const_column_ptr->column_ptr->get_data_at(0);
+ switch (i) {
+ case 1: {
+ std::string field_names_str = const_data.to_string();
+ field_names_str.erase(
+ std::remove_if(field_names_str.begin(), field_names_str.end(),
+ [](unsigned char c) { return std::isspace(c); }),
+ field_names_str.end());
+ std::vector<std::string> field_names;
+ boost::split(field_names, field_names_str, boost::algorithm::is_any_of(","));
+ state->fields.insert(field_names.begin(), field_names.end());
+ } break;
+ case 2:
+ state->type = const_data.to_string();
+ break;
+ case 3:
+ state->query = const_data.to_string();
+ break;
+ default:
+ break;
+ }
+ }
+ }
+
+ return Status::OK();
+}
+
+Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
Review Comment:
warning: function 'eval_inverted_index' exceeds recommended size/complexity thresholds [readability-function-size]
```cpp
Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
^
```
<details>
<summary>Additional context</summary>
**be/src/vec/functions/function_multi_match.cpp:94:** 85 lines including whitespace and comments (threshold 80)
```cpp
Status FunctionMultiMatch::eval_inverted_index(FunctionContext* context,
^
```
</details>
##########
be/src/vec/functions/function_multi_match.h:
##########
@@ -0,0 +1,70 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <boost/algorithm/string/split.hpp>
Review Comment:
warning: 'boost/algorithm/string/split.hpp' file not found [clang-diagnostic-error]
```cpp
#include <boost/algorithm/string/split.hpp>
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](inverted index) multi_match function add [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37722:
URL: https://github.com/apache/doris/pull/37722#issuecomment-2242160750
<details>
<summary>TPC-H: <b>Total hot run time: 40355 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fee667fc11c24a3f915471555fc4e7249ad1970b, data reload: false
------ Round 1 ----------------------------------
q1 18118 4713 4472 4472
q2 2008 194 199 194
q3 10443 1284 1053 1053
q4 10193 837 796 796
q5 7558 2700 2778 2700
q6 216 139 140 139
q7 958 602 625 602
q8 9227 2126 2087 2087
q9 8750 6568 6615 6568
q10 8737 3858 3825 3825
q11 466 244 251 244
q12 395 233 236 233
q13 17753 2987 2989 2987
q14 287 242 250 242
q15 533 486 476 476
q16 484 384 386 384
q17 992 612 703 612
q18 8232 7514 7546 7514
q19 7018 1407 1513 1407
q20 682 323 332 323
q21 5046 3207 3344 3207
q22 352 291 290 290
Total cold run time: 118448 ms
Total hot run time: 40355 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4401 4255 4364 4255
q2 383 272 276 272
q3 3033 2764 2790 2764
q4 1904 1670 1614 1614
q5 5291 5358 5356 5356
q6 215 133 132 132
q7 2141 1702 1749 1702
q8 3208 3369 3328 3328
q9 8505 8395 8423 8395
q10 3931 3684 3665 3665
q11 573 467 486 467
q12 781 635 584 584
q13 16839 2983 3037 2983
q14 299 272 283 272
q15 518 473 485 473
q16 484 438 422 422
q17 1792 1490 1496 1490
q18 7706 7592 7455 7455
q19 1706 1588 1524 1524
q20 2013 1786 1782 1782
q21 4799 4624 4690 4624
q22 573 506 504 504
Total cold run time: 71095 ms
Total hot run time: 54063 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org