You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "DarvenDuan (via GitHub)" <gi...@apache.org> on 2024/03/04 09:30:55 UTC
[PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
DarvenDuan opened a new pull request, #31741:
URL: https://github.com/apache/doris/pull/31741
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
JDBC scan node can use runtime filter to filter data, 2 main changes:
1. Generating runtime filters for JDBC scan node if left table is a JDBC external table in join.
2. Converting IN and MIN_MAX runtime filters to query strings like `col in (v1, v2 . . .)` and `col >= min_v and col <= max_v`, then rebuilding `_query_string` for JDBCScanNode.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982587154
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982219019
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982570383
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982224265
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #31741:
URL: https://github.com/apache/doris/pull/31741#discussion_r1510875372
##########
be/src/vec/exec/scan/new_jdbc_scan_node.h:
##########
@@ -45,6 +45,14 @@ class NewJdbcScanNode : public VScanNode {
protected:
Status _init_profile() override;
Status _init_scanners(std::list<VScannerSPtr>* scanners) override;
+ Status _process_conjuncts() override;
+
+private:
+ Status _process_rf_exprs();
+ Status _convert_rf_expr_to_sql_str(const vectorized::VExprSPtr& expr,
+ std::vector<std::string>& filters);
+ std::string _convert_value_data(PrimitiveType type, const void* value);
+ void _append_rf_string(std::string filter_str);
private:
Review Comment:
warning: redundant access specifier has the same accessibility as the previous access specifier [readability-redundant-access-specifiers]
```suggestion
```
<details>
<summary>Additional context</summary>
**be/src/vec/exec/scan/new_jdbc_scan_node.h:49:** previously declared here
```cpp
private:
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982970673
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1976117984
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017727142
<details>
<summary>TPC-DS: <b>Total hot run time: 185535 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 982f4cfc4da9cdd8641ff426138adcb3f59865d7, data reload: false
query1 932 365 345 345
query2 6561 1914 1801 1801
query3 6716 209 213 209
query4 31700 21266 21254 21254
query5 4331 402 386 386
query6 264 180 171 171
query7 4626 280 282 280
query8 223 177 182 177
query9 9383 2275 2246 2246
query10 560 248 245 245
query11 15511 14176 14183 14176
query12 138 92 87 87
query13 1628 423 411 411
query14 12837 11640 11123 11123
query15 283 196 194 194
query16 8109 264 258 258
query17 1932 567 550 550
query18 2054 292 282 282
query19 276 153 153 153
query20 96 87 89 87
query21 204 126 127 126
query22 4967 4869 4811 4811
query23 33575 32836 32662 32662
query24 10467 2849 2828 2828
query25 611 386 390 386
query26 1246 154 157 154
query27 2441 336 349 336
query28 7196 1856 1838 1838
query29 897 656 638 638
query30 302 151 148 148
query31 968 745 700 700
query32 95 57 54 54
query33 761 253 245 245
query34 1122 467 477 467
query35 827 621 600 600
query36 1030 903 887 887
query37 125 79 78 78
query38 3548 3418 3440 3418
query39 1500 1473 1435 1435
query40 206 114 113 113
query41 53 47 50 47
query42 104 98 94 94
query43 479 460 451 451
query44 1227 706 724 706
query45 270 262 267 262
query46 1101 717 696 696
query47 1912 1852 1839 1839
query48 443 344 344 344
query49 1124 335 330 330
query50 757 373 380 373
query51 6675 6676 6628 6628
query52 111 86 96 86
query53 342 273 285 273
query54 319 233 244 233
query55 85 81 82 81
query56 248 227 225 225
query57 1192 1143 1151 1143
query58 236 202 210 202
query59 2843 2595 2630 2595
query60 260 247 274 247
query61 113 110 109 109
query62 682 454 472 454
query63 311 279 280 279
query64 5814 4022 3998 3998
query65 3056 3066 3046 3046
query66 886 349 353 349
query67 15364 14660 14680 14660
query68 7171 523 515 515
query69 633 382 371 371
query70 1236 1165 1158 1158
query71 540 260 261 260
query72 6618 2708 2554 2554
query73 718 314 317 314
query74 8207 6351 6317 6317
query75 4049 2878 2958 2878
query76 5024 957 877 877
query77 647 255 255 255
query78 10974 10115 10126 10115
query79 9966 511 513 511
query80 1674 365 365 365
query81 536 217 237 217
query82 869 197 192 192
query83 217 139 147 139
query84 281 75 80 75
query85 1460 312 317 312
query86 480 321 301 301
query87 3761 3545 3537 3537
query88 4941 2260 2292 2260
query89 519 365 370 365
query90 1944 169 169 169
query91 168 137 136 136
query92 60 46 48 46
query93 6956 499 483 483
query94 1184 181 174 174
query95 425 322 314 314
query96 605 270 265 265
query97 3074 2866 2893 2866
query98 233 211 201 201
query99 1235 884 924 884
Total cold run time: 312660 ms
Total hot run time: 185535 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991381227
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980694352
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1978477131
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982861505
TeamCity be ut coverage result:
Function Coverage: 35.39% (8561/24189)
Line Coverage: 27.13% (69509/256219)
Region Coverage: 26.36% (36058/136808)
Branch Coverage: 23.18% (18428/79492)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e262b2dc551f4a98131debf97fdd280ab4906680_e262b2dc551f4a98131debf97fdd280ab4906680/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2019311045
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017981267
Could you add this feature on Nereids planner, not legacy planner, please?
legacy planner is planned to be removed in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982221578
run performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980700248
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987610691
<details>
<summary>ClickBench: <b>Total hot run time: 31.52 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4acd0ed6547f3bcd67338b68c4149837ff59e7cf, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.22 0.06 0.07
query4 1.69 0.09 0.10
query5 0.52 0.51 0.51
query6 1.30 0.66 0.67
query7 0.01 0.02 0.01
query8 0.04 0.03 0.04
query9 0.56 0.52 0.54
query10 0.58 0.57 0.57
query11 0.13 0.09 0.10
query12 0.13 0.10 0.10
query13 0.58 0.57 0.57
query14 0.75 0.75 0.78
query15 0.83 0.80 0.81
query16 0.36 0.37 0.37
query17 0.97 0.98 0.94
query18 0.27 0.25 0.26
query19 1.84 1.73 1.72
query20 0.01 0.01 0.01
query21 15.41 0.67 0.67
query22 3.43 4.42 3.17
query23 17.40 1.04 0.97
query24 2.13 0.38 0.38
query25 0.33 0.14 0.02
query26 0.15 0.13 0.13
query27 0.03 0.03 0.04
query28 12.28 0.90 0.83
query29 12.82 3.21 3.32
query30 0.59 0.53 0.57
query31 2.78 0.34 0.33
query32 3.42 0.44 0.43
query33 2.84 2.82 2.85
query34 15.54 4.29 4.29
query35 4.33 4.30 4.32
query36 1.08 1.03 1.02
query37 0.07 0.06 0.05
query38 0.04 0.04 0.03
query39 0.02 0.01 0.02
query40 0.17 0.13 0.13
query41 0.08 0.02 0.02
query42 0.02 0.02 0.02
query43 0.03 0.03 0.03
Total cold run time: 105.88 s
Total hot run time: 31.52 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987602161
TeamCity be ut coverage result:
Function Coverage: 34.99% (8556/24451)
Line Coverage: 26.81% (69529/259327)
Region Coverage: 26.07% (36085/138419)
Branch Coverage: 23.02% (18442/80120)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4acd0ed6547f3bcd67338b68c4149837ff59e7cf_4acd0ed6547f3bcd67338b68c4149837ff59e7cf/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987513427
run buidlall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2076581723
TeamCity be ut coverage result:
Function Coverage: 35.19% (8917/25342)
Line Coverage: 26.95% (73315/272044)
Region Coverage: 26.13% (37883/144955)
Branch Coverage: 22.94% (19287/84058)
Coverage Report: http://coverage.selectdb-in.cc/coverage/982f4cfc4da9cdd8641ff426138adcb3f59865d7_982f4cfc4da9cdd8641ff426138adcb3f59865d7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991099494
<details>
<summary>TPC-H: <b>Total hot run time: 36668 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99b7c0f84af8d01ef1832d020ff408a28569d4d2, data reload: false
------ Round 1 ----------------------------------
q1 17936 4397 4203 4203
q2 2462 151 144 144
q3 11866 934 935 934
q4 4956 710 716 710
q5 7728 2887 2817 2817
q6 190 126 132 126
q7 1196 820 798 798
q8 9493 2073 2034 2034
q9 7286 6489 6423 6423
q10 8107 2474 2531 2474
q11 418 221 210 210
q12 809 281 291 281
q13 17983 3032 2970 2970
q14 280 247 250 247
q15 500 456 453 453
q16 483 387 393 387
q17 946 585 524 524
q18 6625 5840 5940 5840
q19 1571 1405 1474 1405
q20 553 290 302 290
q21 7333 3092 3225 3092
q22 816 306 310 306
Total cold run time: 109537 ms
Total hot run time: 36668 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4105 4044 4073 4044
q2 320 227 226 226
q3 2992 2799 2811 2799
q4 1794 1535 1563 1535
q5 5204 5257 5206 5206
q6 195 116 118 116
q7 2257 1851 1823 1823
q8 3153 3288 3287 3287
q9 8553 8537 8599 8537
q10 6194 3682 3652 3652
q11 521 423 438 423
q12 722 548 547 547
q13 5593 2887 2877 2877
q14 264 243 247 243
q15 475 470 440 440
q16 442 399 399 399
q17 1717 1470 1447 1447
q18 7582 7243 6996 6996
q19 4188 1497 1550 1497
q20 1925 1689 1685 1685
q21 4814 4696 4722 4696
q22 512 460 483 460
Total cold run time: 63522 ms
Total hot run time: 52935 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017732939
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 982f4cfc4da9cdd8641ff426138adcb3f59865d7 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1977835863
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980772373
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982744665
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1978364777
TeamCity be ut coverage result:
Function Coverage: 35.42% (8556/24157)
Line Coverage: 27.17% (69440/255589)
Region Coverage: 26.39% (36031/136509)
Branch Coverage: 23.22% (18411/79292)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3e6ab743dd51add1a4ef28d63d36007a3d4713d8_3e6ab743dd51add1a4ef28d63d36007a3d4713d8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980421316
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1994043572
Hi, Mr.@xiaokang, could you please review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1983135203
TeamCity be ut coverage result:
Function Coverage: 35.39% (8561/24189)
Line Coverage: 27.13% (69508/256218)
Region Coverage: 26.36% (36056/136808)
Branch Coverage: 23.18% (18427/79492)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b90dd6acdd7f49a545945f4cc2abfa0c783e2f99_b90dd6acdd7f49a545945f4cc2abfa0c783e2f99/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017678152
TeamCity be ut coverage result:
Function Coverage: 35.25% (8733/24774)
Line Coverage: 27.04% (71516/264530)
Region Coverage: 26.27% (37096/141206)
Branch Coverage: 23.16% (18975/81924)
Coverage Report: http://coverage.selectdb-in.cc/coverage/982f4cfc4da9cdd8641ff426138adcb3f59865d7_982f4cfc4da9cdd8641ff426138adcb3f59865d7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1976196452
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982533646
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017642703
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2017706767
<details>
<summary>TPC-H: <b>Total hot run time: 38382 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 982f4cfc4da9cdd8641ff426138adcb3f59865d7, data reload: false
------ Round 1 ----------------------------------
q1 17724 4500 4163 4163
q2 2116 160 151 151
q3 10588 1147 1211 1147
q4 10217 781 819 781
q5 7477 3034 2990 2990
q6 203 125 122 122
q7 1047 570 568 568
q8 9339 2010 2007 2007
q9 7497 6677 6682 6677
q10 8540 3602 3663 3602
q11 425 227 233 227
q12 427 203 197 197
q13 17795 2839 2861 2839
q14 229 205 199 199
q15 503 468 467 467
q16 500 369 364 364
q17 978 532 547 532
q18 7279 6537 6422 6422
q19 2603 1520 1405 1405
q20 546 246 244 244
q21 3563 3012 2966 2966
q22 357 322 312 312
Total cold run time: 109953 ms
Total hot run time: 38382 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4188 4104 4072 4072
q2 321 234 229 229
q3 3004 2833 2843 2833
q4 1841 1581 1563 1563
q5 5302 5358 5313 5313
q6 193 120 117 117
q7 2212 1817 1840 1817
q8 3164 3313 3281 3281
q9 8689 8662 8730 8662
q10 3813 3740 3815 3740
q11 544 460 443 443
q12 724 539 527 527
q13 16921 2869 2841 2841
q14 281 259 262 259
q15 509 466 467 466
q16 472 419 428 419
q17 1791 1542 1473 1473
q18 7577 7117 7216 7117
q19 1639 1534 1457 1457
q20 1922 1706 1717 1706
q21 4839 4773 4798 4773
q22 497 446 466 446
Total cold run time: 70443 ms
Total hot run time: 53554 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991150514
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 99b7c0f84af8d01ef1832d020ff408a28569d4d2 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 16.6 seconds inserted 10000000 Rows, about 602K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991390410
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1978045826
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1978023849
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982221259
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982715982
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991055834
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991189674
TeamCity be ut coverage result:
Function Coverage: 34.97% (8556/24464)
Line Coverage: 26.80% (69533/259469)
Region Coverage: 26.05% (36080/138508)
Branch Coverage: 23.02% (18452/80168)
Coverage Report: http://coverage.selectdb-in.cc/coverage/99b7c0f84af8d01ef1832d020ff408a28569d4d2_99b7c0f84af8d01ef1832d020ff408a28569d4d2/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2076538226
run beut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2430541382
We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1983164000
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982520502
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982222613
run cload_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987606839
<details>
<summary>TPC-DS: <b>Total hot run time: 185485 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4acd0ed6547f3bcd67338b68c4149837ff59e7cf, data reload: false
query1 911 359 339 339
query2 7416 1827 1998 1827
query3 6702 208 206 206
query4 24453 20968 21122 20968
query5 4264 576 459 459
query6 275 178 167 167
query7 4626 311 302 302
query8 236 167 165 165
query9 8534 2171 2164 2164
query10 459 237 247 237
query11 14963 14537 14439 14439
query12 137 93 90 90
query13 1651 451 437 437
query14 10585 9993 9889 9889
query15 244 201 204 201
query16 7752 260 255 255
query17 1908 567 537 537
query18 1946 279 273 273
query19 200 151 156 151
query20 98 104 84 84
query21 196 126 121 121
query22 4564 4467 4416 4416
query23 31271 30689 30869 30689
query24 10748 3088 3059 3059
query25 653 392 400 392
query26 1009 164 170 164
query27 2441 374 359 359
query28 6313 1808 1790 1790
query29 1005 607 596 596
query30 321 148 148 148
query31 927 721 732 721
query32 108 64 61 61
query33 753 277 246 246
query34 1022 464 469 464
query35 855 640 640 640
query36 923 812 809 809
query37 124 74 76 74
query38 3271 3139 3150 3139
query39 1441 1413 1380 1380
query40 214 117 114 114
query41 69 64 60 60
query42 105 103 104 103
query43 422 404 399 399
query44 1074 711 706 706
query45 257 255 259 255
query46 1050 776 750 750
query47 1650 1555 1569 1555
query48 417 361 347 347
query49 1132 326 345 326
query50 778 373 370 370
query51 6661 6534 6627 6534
query52 113 101 93 93
query53 351 279 284 279
query54 334 240 258 240
query55 84 85 83 83
query56 238 229 230 229
query57 1089 1005 999 999
query58 240 214 216 214
query59 2455 2270 2269 2269
query60 273 254 297 254
query61 112 109 107 107
query62 654 399 414 399
query63 306 279 274 274
query64 5177 3317 3132 3132
query65 3007 2995 2975 2975
query66 861 323 333 323
query67 15062 14301 14192 14192
query68 6818 551 552 551
query69 618 390 382 382
query70 1170 1117 1093 1093
query71 540 288 279 279
query72 6468 2626 2533 2533
query73 779 325 318 318
query74 6833 6523 6495 6495
query75 9192 8070 7950 7950
query76 4909 1123 1170 1123
query77 662 278 253 253
query78 10127 9602 9614 9602
query79 8584 524 513 513
query80 1598 395 376 376
query81 522 217 212 212
query82 789 208 198 198
query83 202 143 143 143
query84 285 79 76 76
query85 1463 348 316 316
query86 464 298 288 288
query87 3504 3272 3234 3234
query88 4165 2302 2287 2287
query89 544 361 367 361
query90 1970 182 171 171
query91 156 152 147 147
query92 62 48 50 48
query93 6826 523 508 508
query94 1114 180 182 180
query95 445 337 343 337
query96 590 266 278 266
query97 4389 4257 4287 4257
query98 225 214 199 199
query99 1187 784 750 750
Total cold run time: 296690 ms
Total hot run time: 185485 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2019842755
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987554700
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #31741: [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter
URL: https://github.com/apache/doris/pull/31741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1987598889
<details>
<summary>TPC-H: <b>Total hot run time: 38439 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4acd0ed6547f3bcd67338b68c4149837ff59e7cf, data reload: false
------ Round 1 ----------------------------------
q1 17871 4015 4013 4013
q2 2016 148 158 148
q3 10702 974 929 929
q4 4666 956 945 945
q5 8042 2886 2992 2886
q6 178 125 127 125
q7 1302 820 826 820
q8 9476 2090 2072 2072
q9 7405 6547 6514 6514
q10 8282 2579 2550 2550
q11 422 215 212 212
q12 798 325 312 312
q13 18049 3029 3113 3029
q14 289 281 280 280
q15 506 480 461 461
q16 488 416 416 416
q17 947 873 891 873
q18 8403 6476 6099 6099
q19 1559 1520 1507 1507
q20 566 281 277 277
q21 7727 3678 3710 3678
q22 2503 293 349 293
Total cold run time: 112197 ms
Total hot run time: 38439 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4017 3976 3941 3941
q2 312 223 225 223
q3 2950 2927 2863 2863
q4 1800 1820 1783 1783
q5 5205 5258 5237 5237
q6 200 116 118 116
q7 2266 1826 1802 1802
q8 3176 3230 3259 3230
q9 8472 8507 8508 8507
q10 6153 3706 3699 3699
q11 526 444 437 437
q12 712 561 524 524
q13 10671 2863 2861 2861
q14 271 256 257 256
q15 475 449 445 445
q16 451 426 411 411
q17 1667 1665 1646 1646
q18 7808 7438 7229 7229
q19 4670 1486 1590 1486
q20 1882 1689 1693 1689
q21 5039 4839 4776 4776
q22 498 442 431 431
Total cold run time: 69221 ms
Total hot run time: 53592 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982222828
run cloud_p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1982222475
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991144650
<details>
<summary>ClickBench: <b>Total hot run time: 30.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99b7c0f84af8d01ef1832d020ff408a28569d4d2, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.03 0.02
query3 0.23 0.06 0.06
query4 1.65 0.10 0.10
query5 0.52 0.49 0.52
query6 1.18 0.62 0.63
query7 0.02 0.01 0.02
query8 0.03 0.03 0.03
query9 0.56 0.49 0.48
query10 0.56 0.57 0.56
query11 0.12 0.09 0.10
query12 0.11 0.09 0.10
query13 0.58 0.56 0.55
query14 0.74 0.74 0.74
query15 0.79 0.78 0.77
query16 0.39 0.39 0.39
query17 0.99 1.02 0.96
query18 0.20 0.23 0.25
query19 1.77 1.70 1.76
query20 0.02 0.01 0.01
query21 15.44 0.56 0.54
query22 2.37 2.17 2.14
query23 17.10 0.80 0.77
query24 3.10 2.21 1.28
query25 0.19 0.13 0.04
query26 0.85 0.13 0.12
query27 0.03 0.03 0.03
query28 10.09 0.87 0.85
query29 12.53 3.19 3.14
query30 0.67 0.54 0.56
query31 2.82 0.34 0.33
query32 3.36 0.43 0.43
query33 2.76 2.85 2.80
query34 16.18 4.31 4.32
query35 4.30 4.31 4.34
query36 1.07 1.01 1.00
query37 0.08 0.05 0.06
query38 0.04 0.03 0.03
query39 0.03 0.01 0.01
query40 0.17 0.14 0.14
query41 0.08 0.02 0.02
query42 0.04 0.02 0.02
query43 0.03 0.03 0.03
Total cold run time: 103.9 s
Total hot run time: 30.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1991132025
<details>
<summary>TPC-DS: <b>Total hot run time: 186936 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 99b7c0f84af8d01ef1832d020ff408a28569d4d2, data reload: false
query1 917 363 333 333
query2 7410 2033 2020 2020
query3 6694 214 208 208
query4 31158 20580 20677 20580
query5 4405 476 472 472
query6 261 175 183 175
query7 4645 301 284 284
query8 233 177 162 162
query9 9899 2308 2295 2295
query10 457 231 240 231
query11 14541 14079 14175 14079
query12 128 87 83 83
query13 1626 412 434 412
query14 11213 10861 10857 10857
query15 297 211 189 189
query16 8062 263 264 263
query17 1941 586 535 535
query18 2083 287 287 287
query19 230 155 158 155
query20 90 107 85 85
query21 202 133 124 124
query22 4509 4462 4390 4390
query23 31755 30967 31093 30967
query24 11766 2843 2825 2825
query25 664 375 387 375
query26 1844 159 165 159
query27 3137 366 348 348
query28 8160 1931 1915 1915
query29 1045 654 636 636
query30 315 146 148 146
query31 970 728 726 726
query32 90 56 53 53
query33 762 256 242 242
query34 1155 468 484 468
query35 832 608 612 608
query36 975 884 856 856
query37 291 84 80 80
query38 3295 3108 3152 3108
query39 1436 1360 1391 1360
query40 296 112 111 111
query41 55 47 45 45
query42 103 100 111 100
query43 479 476 455 455
query44 1160 729 699 699
query45 276 238 248 238
query46 1075 673 672 672
query47 1661 1594 1613 1594
query48 453 354 366 354
query49 1243 341 323 323
query50 745 375 368 368
query51 6660 6631 6577 6577
query52 98 91 114 91
query53 336 266 275 266
query54 327 229 228 228
query55 82 80 80 80
query56 237 214 219 214
query57 1093 999 995 995
query58 240 197 220 197
query59 2860 2580 2778 2580
query60 293 239 258 239
query61 99 93 91 91
query62 599 398 389 389
query63 317 269 272 269
query64 6594 3766 3733 3733
query65 3076 3007 2987 2987
query66 1378 350 319 319
query67 15022 14292 14706 14292
query68 5303 516 531 516
query69 585 384 411 384
query70 1234 1191 1151 1151
query71 465 268 269 268
query72 6707 2676 2626 2626
query73 705 327 317 317
query74 8013 6533 6566 6533
query75 8676 8021 8004 8004
query76 3852 899 889 889
query77 604 260 258 258
query78 10371 9508 9640 9508
query79 8883 522 528 522
query80 1951 407 462 407
query81 545 218 209 209
query82 1340 206 209 206
query83 325 153 149 149
query84 288 81 80 80
query85 1830 373 345 345
query86 469 292 314 292
query87 3424 3170 3244 3170
query88 5099 2347 2387 2347
query89 573 370 376 370
query90 1983 179 177 177
query91 161 128 131 128
query92 64 48 45 45
query93 6945 518 492 492
query94 1221 175 176 175
query95 428 338 317 317
query96 623 270 275 270
query97 4385 4240 4296 4240
query98 228 208 201 201
query99 1139 752 743 743
Total cold run time: 314769 ms
Total hot run time: 186936 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-2019236899
> Could you add this feature on Nereids planner, not legacy planner, please? legacy planner is planned to be removed in future.
This feature on Nereids planner has already been implemented before this PR, we just need to implement it in legacy planner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980476395
TeamCity be ut coverage result:
Function Coverage: 35.40% (8559/24179)
Line Coverage: 27.14% (69481/256002)
Region Coverage: 26.36% (36041/136716)
Branch Coverage: 23.19% (18420/79428)
Coverage Report: http://coverage.selectdb-in.cc/coverage/919e337c3737b6600be4a8eae6ef3da825ddcaf7_919e337c3737b6600be4a8eae6ef3da825ddcaf7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter] (JDBC) jdbc scan node support IN/MIN_MAX runtime filter [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #31741:
URL: https://github.com/apache/doris/pull/31741#issuecomment-1980698671
run performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org