You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "seawinde (via GitHub)" <gi...@apache.org> on 2024/01/14 08:55:55 UTC
[PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
seawinde opened a new pull request, #29953:
URL: https://github.com/apache/doris/pull/29953
## Proposed changes
1. Make sure instance when change params of StructInfo,Predicates.
2. Catch and record exception for every materialization context, this make sure that if throw exception when one materialization context rewrite, it will not influence others.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891594508
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ddacbde5f3c406be66a3eb10ab3ee2a45cf10edd with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.7 seconds inserted 10000000 Rows, about 680K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894139123
<details>
<summary>TPC-H: <b>Total hot run time: 38912 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit af4788261e9536e2128a00befe9ff4be6c1114ff, data reload: false
------ Round 1 ----------------------------------
q1 10673 5466 5266 5266
q2 1288 147 133 133
q3 6367 1181 1162 1162
q4 1025 755 771 755
q5 3315 3261 3061 3061
q6 206 125 123 123
q7 886 487 484 484
q8 1845 1904 1926 1904
q9 6416 6352 6322 6322
q10 3081 3054 3044 3044
q11 331 228 220 220
q12 354 201 198 198
q13 17749 3380 3368 3368
q14 244 215 225 215
q15 542 505 506 505
q16 437 398 409 398
q17 927 455 521 455
q18 7756 7052 6883 6883
q19 1511 1367 1386 1367
q20 470 288 325 288
q21 2785 2443 2496 2443
q22 348 318 323 318
Total cold run time: 68556 ms
Total hot run time: 38912 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5392 5381 5319 5319
q2 327 215 216 215
q3 3339 3245 3248 3245
q4 2099 2035 2050 2035
q5 6140 5790 5872 5790
q6 203 122 124 122
q7 2297 1875 1984 1875
q8 3231 3399 3394 3394
q9 8903 9062 8905 8905
q10 9958 3762 3818 3762
q11 571 448 458 448
q12 973 637 639 637
q13 3932 3250 3200 3200
q14 284 253 270 253
q15 593 510 503 503
q16 537 503 487 487
q17 2512 1786 1810 1786
q18 9641 9198 9562 9198
q19 20959 1581 1474 1474
q20 4462 1971 1939 1939
q21 14441 5525 5468 5468
q22 1005 537 553 537
Total cold run time: 101799 ms
Total hot run time: 60592 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891561896
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.1 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.3 seconds inserted 10000000 Rows, about 469K ops/s
storage size: 17183914930 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1899961192
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1900029214
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit cb880fafdd0764de2e2b4f5e55cc4bc42401dc2a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.6 seconds inserted 10000000 Rows, about 684K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1897888507
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on code in PR #29953:
URL: https://github.com/apache/doris/pull/29953#discussion_r1453685479
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/StructInfo.java:
##########
@@ -216,11 +226,39 @@ public static List<StructInfo> of(Plan originalPlan) {
List<HyperGraph> structInfos = HyperGraph.toStructInfo(planSplitContext.getBottomPlan());
return structInfos.stream()
- .map(hyperGraph -> new StructInfo(originalPlan, planSplitContext.getTopPlan(),
+ .map(hyperGraph -> StructInfo.of(originalPlan, planSplitContext.getTopPlan(),
planSplitContext.getBottomPlan(), hyperGraph))
.collect(Collectors.toList());
}
+ /**
+ * The construct method for init StructInfo
+ */
+ public static StructInfo of(Plan originalPlan, @Nullable Plan topPlan, @Nullable Plan bottomPlan,
+ HyperGraph hyperGraph) {
+ ObjectId originalPlanId = originalPlan.getGroupExpression()
+ .map(GroupExpression::getId).orElseGet(() -> new ObjectId(-1));
+ // if any of topPlan or bottomPlan is null, split the top plan to two parts by join node
+ if (topPlan == null || bottomPlan == null) {
+ PlanSplitContext planSplitContext = new PlanSplitContext(Sets.newHashSet(LogicalJoin.class));
+ originalPlan.accept(PLAN_SPLITTER, planSplitContext);
+ bottomPlan = planSplitContext.getBottomPlan();
+ topPlan = planSplitContext.getTopPlan();
+ }
+ // collect struct info fromGraph
+ List<CatalogRelation> relations = new ArrayList<>();
+ Map<RelationId, StructInfoNode> relationIdStructInfoNodeMap = new HashMap<>();
+ Map<Expression, Expression> shuttledHashConjunctsToConjunctsMap = new HashMap<>();
+ Map<ExprId, Expression> namedExprIdAndExprMapping = new HashMap<>();
+ boolean valid = collectStructInfoFromGraph(hyperGraph, topPlan, shuttledHashConjunctsToConjunctsMap,
+ namedExprIdAndExprMapping,
+ relations,
+ relationIdStructInfoNodeMap);
Review Comment:
Have fixed it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894105223
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1899964589
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891585761
<details>
<summary>ClickBench: <b>Total hot run time: 31.27 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ddacbde5f3c406be66a3eb10ab3ee2a45cf10edd, data reload: false
query1 0.06 0.06 0.06
query2 0.06 0.02 0.02
query3 0.25 0.12 0.11
query4 1.78 0.13 0.12
query5 0.53 0.53 0.54
query6 1.31 0.64 0.64
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.56 0.51 0.51
query10 0.56 0.56 0.55
query11 0.12 0.10 0.09
query12 0.11 0.10 0.09
query13 0.61 0.60 0.60
query14 0.78 0.78 0.80
query15 0.80 0.77 0.77
query16 0.38 0.38 0.38
query17 1.01 1.01 1.03
query18 0.24 0.25 0.27
query19 1.81 1.79 1.77
query20 0.01 0.01 0.00
query21 15.40 0.58 0.55
query22 2.71 2.16 2.59
query23 17.03 0.88 0.83
query24 2.56 1.24 0.68
query25 0.44 0.21 0.20
query26 0.44 0.16 0.15
query27 0.05 0.06 0.05
query28 11.90 0.79 0.80
query29 12.58 3.25 3.28
query30 0.65 0.53 0.54
query31 2.80 0.34 0.35
query32 3.39 0.48 0.48
query33 3.22 3.25 3.21
query34 15.82 4.18 4.21
query35 4.23 4.28 4.24
query36 1.13 1.07 1.08
query37 0.06 0.05 0.05
query38 0.04 0.02 0.04
query39 0.02 0.01 0.01
query40 0.16 0.14 0.13
query41 0.07 0.01 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 105.79 s
Total hot run time: 31.27 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1892163700
<details>
<summary>TPC-DS: <b>Total hot run time: 179709 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 774a11e5f5ccc49b7bff6b4a277529b928064250, data reload: false
query1 939 346 325 325
query2 6721 1831 1857 1831
query3 6743 215 206 206
query4 33742 22188 22107 22107
query5 6949 555 602 555
query6 269 198 194 194
query7 4613 284 284 284
query8 231 201 196 196
query9 9138 2664 2767 2664
query10 642 252 236 236
query11 16504 15250 15281 15250
query12 123 74 74 74
query13 1693 383 391 383
query14 11376 7623 7382 7382
query15 209 195 192 192
query16 4633 257 241 241
query17 1000 506 489 489
query18 1807 274 267 267
query19 194 156 150 150
query20 80 76 71 71
query21 191 150 134 134
query22 4970 4912 4974 4912
query23 32176 31255 31142 31142
query24 12520 2782 2868 2782
query25 578 330 336 330
query26 1830 156 158 156
query27 3220 282 284 282
query28 7317 1877 1857 1857
query29 2221 713 692 692
query30 285 141 144 141
query31 909 748 775 748
query32 72 61 57 57
query33 715 243 261 243
query34 1147 474 472 472
query35 898 778 787 778
query36 1415 1302 1351 1302
query37 183 70 75 70
query38 3443 3266 3279 3266
query39 1331 1276 1280 1276
query40 344 101 96 96
query41 39 36 35 35
query42 107 97 99 97
query43 527 496 520 496
query44 1111 709 729 709
query45 204 188 183 183
query46 1079 670 648 648
query47 1675 1584 1570 1570
query48 445 325 320 320
query49 1245 327 312 312
query50 708 320 331 320
query51 5406 5276 5288 5276
query52 104 93 81 81
query53 371 292 287 287
query54 916 447 467 447
query55 101 88 88 88
query56 216 191 201 191
query57 1024 959 958 958
query58 221 197 201 197
query59 2968 2659 2669 2659
query60 248 221 216 216
query61 85 104 90 90
query62 649 424 392 392
query63 310 292 296 292
query64 6295 1738 1736 1736
query65 3387 3294 3266 3266
query66 1421 333 325 325
query67 15768 15238 15784 15238
query68 11961 533 513 513
query69 657 345 364 345
query70 1745 1617 1620 1617
query71 567 243 254 243
query72 5172 2864 2948 2864
query73 2619 329 325 325
query74 6906 6633 6387 6387
query75 5208 2353 2306 2306
query76 6298 968 1004 968
query77 802 272 283 272
query78 9508 8759 8680 8680
query79 1049 529 518 518
query80 582 340 357 340
query81 442 215 206 206
query82 208 96 92 92
query83 140 124 126 124
query84 273 73 73 73
query85 1062 331 326 326
query86 395 410 416 410
query87 3568 3401 3381 3381
query88 3224 2223 2226 2223
query89 454 388 400 388
query90 2009 220 200 200
query91 154 122 125 122
query92 66 59 52 52
query93 1417 440 438 438
query94 1210 186 176 176
query95 540 490 475 475
query96 643 328 325 325
query97 4306 4155 4155 4155
query98 218 193 191 191
query99 1078 712 741 712
Total cold run time: 304482 ms
Total hot run time: 179709 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890904912
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.46 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
storage size: 17183914998 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1900016456
<details>
<summary>TPC-DS: <b>Total hot run time: 176283 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb880fafdd0764de2e2b4f5e55cc4bc42401dc2a, data reload: false
query1 947 336 327 327
query2 6555 1836 1855 1836
query3 6717 199 196 196
query4 30877 22261 22212 22212
query5 6871 535 598 535
query6 256 152 154 152
query7 4645 261 262 261
query8 233 182 176 176
query9 8364 2539 2530 2530
query10 414 225 222 222
query11 16331 15591 15342 15342
query12 131 70 66 66
query13 1680 370 372 370
query14 10605 7004 6948 6948
query15 211 182 181 181
query16 4596 233 226 226
query17 997 468 477 468
query18 1790 267 251 251
query19 184 131 130 130
query20 68 78 74 74
query21 193 126 129 126
query22 4951 4834 4762 4762
query23 31576 30856 30637 30637
query24 12976 2795 2814 2795
query25 573 313 302 302
query26 1839 137 141 137
query27 3455 268 272 268
query28 7183 1821 1818 1818
query29 2161 637 613 613
query30 283 137 137 137
query31 958 722 758 722
query32 75 52 51 51
query33 710 215 211 211
query34 1315 447 466 447
query35 889 788 790 788
query36 1366 1176 1233 1176
query37 174 64 59 59
query38 3342 3260 3213 3213
query39 1307 1277 1260 1260
query40 342 89 84 84
query41 40 35 35 35
query42 94 84 87 84
query43 526 471 449 449
query44 1105 691 688 688
query45 189 174 173 173
query46 1062 654 639 639
query47 1628 1567 1517 1517
query48 379 304 326 304
query49 1220 283 278 278
query50 687 301 313 301
query51 5312 5182 5236 5182
query52 92 76 74 74
query53 332 253 253 253
query54 457 282 288 282
query55 84 73 75 73
query56 180 168 172 168
query57 990 922 934 922
query58 188 160 160 160
query59 2938 2538 2612 2538
query60 202 195 192 192
query61 80 84 81 81
query62 601 354 365 354
query63 288 260 260 260
query64 6223 1768 1760 1760
query65 3324 3249 3230 3230
query66 1432 316 316 316
query67 15386 15248 15006 15006
query68 13468 534 501 501
query69 609 294 296 294
query70 2082 1524 1500 1500
query71 501 212 211 211
query72 5049 2847 2821 2821
query73 3681 310 306 306
query74 6908 6399 6418 6399
query75 4849 2346 2310 2310
query76 6200 1057 971 971
query77 679 226 229 226
query78 9026 8766 8763 8763
query79 1945 496 515 496
query80 531 326 314 314
query81 453 207 204 204
query82 204 81 82 81
query83 135 126 121 121
query84 273 72 69 69
query85 1070 330 314 314
query86 396 396 424 396
query87 3511 3356 3350 3350
query88 2927 2190 2188 2188
query89 441 369 354 354
query90 1949 186 179 179
query91 151 129 139 129
query92 53 46 47 46
query93 1364 418 435 418
query94 1217 157 155 155
query95 507 438 451 438
query96 623 316 320 316
query97 4294 4187 4129 4129
query98 200 190 187 187
query99 992 733 669 669
Total cold run time: 299467 ms
Total hot run time: 176283 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890906537
<details>
<summary>TPC-DS: <b>Total hot run time: 179651 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 598851e65984d583a4da70876e4992e67c0451b5, data reload: false
query1 917 337 328 328
query2 6746 1830 1943 1830
query3 6701 222 216 216
query4 34755 22159 22092 22092
query5 6974 550 605 550
query6 257 180 184 180
query7 4617 272 267 267
query8 247 220 200 200
query9 9078 2600 2751 2600
query10 605 237 237 237
query11 16600 15419 15318 15318
query12 128 72 71 71
query13 1704 372 396 372
query14 11281 7293 7417 7293
query15 226 194 191 191
query16 4664 252 257 252
query17 1001 495 481 481
query18 1800 274 261 261
query19 184 156 146 146
query20 82 75 76 75
query21 199 141 126 126
query22 5124 5108 5085 5085
query23 32223 31342 31617 31342
query24 12992 2896 2905 2896
query25 598 349 361 349
query26 1810 168 163 163
query27 3345 318 297 297
query28 8365 1876 1897 1876
query29 3401 874 730 730
query30 741 145 147 145
query31 1806 774 794 774
query32 121 63 59 59
query33 1487 267 243 243
query34 1348 466 477 466
query35 908 802 790 790
query36 1618 1306 1299 1299
query37 226 71 96 71
query38 3418 3265 3252 3252
query39 1329 1289 1281 1281
query40 344 97 96 96
query41 37 35 35 35
query42 105 91 99 91
query43 542 504 531 504
query44 1098 694 713 694
query45 201 197 181 181
query46 1075 652 649 649
query47 1708 1569 1538 1538
query48 389 303 327 303
query49 1253 313 318 313
query50 696 322 329 322
query51 5424 5224 5274 5224
query52 102 88 101 88
query53 365 288 297 288
query54 926 450 454 450
query55 92 94 91 91
query56 209 199 196 196
query57 1011 951 929 929
query58 220 190 189 189
query59 2982 2659 2601 2601
query60 248 230 222 222
query61 86 80 86 80
query62 671 434 400 400
query63 321 292 289 289
query64 6376 1709 1648 1648
query65 3353 3263 3268 3263
query66 1431 331 326 326
query67 15813 15258 15299 15258
query68 10221 510 530 510
query69 659 343 347 343
query70 1731 1513 1591 1513
query71 564 231 249 231
query72 4763 2854 2877 2854
query73 2091 320 325 320
query74 7144 6455 6478 6455
query75 4880 2291 2335 2291
query76 6315 923 1082 923
query77 930 264 288 264
query78 9124 8913 8582 8582
query79 1022 523 493 493
query80 535 337 341 337
query81 446 211 210 210
query82 233 97 95 95
query83 144 126 127 126
query84 279 73 71 71
query85 1056 333 313 313
query86 388 373 368 368
query87 3529 3368 3361 3361
query88 2856 2271 2268 2268
query89 454 392 388 388
query90 2083 208 204 204
query91 159 136 125 125
query92 61 57 54 54
query93 951 407 401 401
query94 1172 187 182 182
query95 514 476 473 473
query96 630 329 338 329
query97 4293 4189 4188 4188
query98 209 196 192 192
query99 1010 715 747 715
Total cold run time: 307150 ms
Total hot run time: 179651 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891264552
<details>
<summary>ClickBench: <b>Total hot run time: 31.51 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1000e47f26f772a5591f1a215e7317d2d6e9803f, data reload: false
query1 0.05 0.06 0.06
query2 0.05 0.02 0.02
query3 0.25 0.11 0.11
query4 1.80 0.12 0.11
query5 0.53 0.52 0.52
query6 1.30 0.65 0.63
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.55 0.51 0.51
query10 0.56 0.55 0.54
query11 0.12 0.09 0.09
query12 0.12 0.10 0.09
query13 0.61 0.60 0.60
query14 0.78 0.77 0.78
query15 0.80 0.77 0.77
query16 0.40 0.37 0.36
query17 1.04 1.01 1.03
query18 0.24 0.27 0.26
query19 1.82 1.69 1.77
query20 0.02 0.01 0.01
query21 15.41 0.58 0.56
query22 2.49 2.49 2.38
query23 17.12 0.76 0.88
query24 2.60 1.30 0.98
query25 1.98 0.16 0.15
query26 0.14 0.14 0.14
query27 0.15 0.16 0.15
query28 11.25 0.81 0.77
query29 12.57 3.14 3.22
query30 0.58 0.49 0.50
query31 2.78 0.34 0.36
query32 3.38 0.49 0.48
query33 3.23 3.22 3.22
query34 15.91 4.24 4.18
query35 4.20 4.24 4.26
query36 1.13 1.10 1.08
query37 0.06 0.05 0.05
query38 0.03 0.03 0.03
query39 0.02 0.02 0.02
query40 0.16 0.13 0.13
query41 0.07 0.02 0.01
query42 0.03 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 106.4 s
Total hot run time: 31.51 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #29953:
URL: https://github.com/apache/doris/pull/29953#discussion_r1451903864
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/StructInfo.java:
##########
@@ -216,11 +226,39 @@ public static List<StructInfo> of(Plan originalPlan) {
List<HyperGraph> structInfos = HyperGraph.toStructInfo(planSplitContext.getBottomPlan());
return structInfos.stream()
- .map(hyperGraph -> new StructInfo(originalPlan, planSplitContext.getTopPlan(),
+ .map(hyperGraph -> StructInfo.of(originalPlan, planSplitContext.getTopPlan(),
planSplitContext.getBottomPlan(), hyperGraph))
.collect(Collectors.toList());
}
+ /**
+ * The construct method for init StructInfo
+ */
+ public static StructInfo of(Plan originalPlan, @Nullable Plan topPlan, @Nullable Plan bottomPlan,
+ HyperGraph hyperGraph) {
+ ObjectId originalPlanId = originalPlan.getGroupExpression()
+ .map(GroupExpression::getId).orElseGet(() -> new ObjectId(-1));
+ // if any of topPlan or bottomPlan is null, split the top plan to two parts by join node
+ if (topPlan == null || bottomPlan == null) {
+ PlanSplitContext planSplitContext = new PlanSplitContext(Sets.newHashSet(LogicalJoin.class));
+ originalPlan.accept(PLAN_SPLITTER, planSplitContext);
+ bottomPlan = planSplitContext.getBottomPlan();
+ topPlan = planSplitContext.getTopPlan();
+ }
+ // collect struct info fromGraph
+ List<CatalogRelation> relations = new ArrayList<>();
+ Map<RelationId, StructInfoNode> relationIdStructInfoNodeMap = new HashMap<>();
+ Map<Expression, Expression> shuttledHashConjunctsToConjunctsMap = new HashMap<>();
+ Map<ExprId, Expression> namedExprIdAndExprMapping = new HashMap<>();
+ boolean valid = collectStructInfoFromGraph(hyperGraph, topPlan, shuttledHashConjunctsToConjunctsMap,
+ namedExprIdAndExprMapping,
+ relations,
+ relationIdStructInfoNodeMap);
Review Comment:
Maybe can use ImmutableList
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1899973538
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894168476
<details>
<summary>ClickBench: <b>Total hot run time: 30.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit af4788261e9536e2128a00befe9ff4be6c1114ff, data reload: false
query1 0.03 0.03 0.03
query2 0.05 0.02 0.03
query3 0.22 0.05 0.05
query4 1.70 0.08 0.11
query5 0.54 0.52 0.52
query6 1.24 0.62 0.63
query7 0.02 0.02 0.01
query8 0.03 0.02 0.03
query9 0.55 0.50 0.50
query10 0.56 0.56 0.56
query11 0.12 0.09 0.09
query12 0.10 0.09 0.09
query13 0.60 0.61 0.59
query14 0.77 0.77 0.79
query15 0.80 0.78 0.78
query16 0.37 0.39 0.37
query17 1.05 1.01 1.01
query18 0.25 0.25 0.25
query19 1.88 1.75 1.79
query20 0.01 0.00 0.01
query21 15.43 0.58 0.56
query22 2.62 2.81 1.71
query23 17.25 0.90 0.88
query24 2.56 0.64 1.45
query25 0.35 0.22 0.13
query26 0.55 0.13 0.12
query27 0.06 0.05 0.05
query28 11.52 0.78 0.76
query29 12.50 3.19 3.31
query30 0.57 0.47 0.48
query31 2.78 0.35 0.34
query32 3.38 0.48 0.49
query33 3.23 3.21 3.21
query34 15.77 4.25 4.30
query35 4.24 4.45 4.24
query36 1.11 1.07 1.08
query37 0.06 0.05 0.05
query38 0.04 0.02 0.02
query39 0.02 0.01 0.01
query40 0.16 0.12 0.13
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.2 s
Total hot run time: 30.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891266420
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1000e47f26f772a5591f1a215e7317d2d6e9803f with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.7 seconds inserted 10000000 Rows, about 680K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1892145168
<details>
<summary>TPC-H: <b>Total hot run time: 40043 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 774a11e5f5ccc49b7bff6b4a277529b928064250, data reload: false
------ Round 1 ----------------------------------
q1 18864 6336 5761 5761
q2 2427 172 137 137
q3 11139 1153 1177 1153
q4 10683 790 787 787
q5 8176 3241 3211 3211
q6 208 124 124 124
q7 932 509 478 478
q8 9270 2022 2018 2018
q9 7356 6441 6427 6427
q10 8232 3096 3075 3075
q11 425 213 224 213
q12 359 194 196 194
q13 18105 3468 3443 3443
q14 258 211 217 211
q15 560 527 516 516
q16 449 378 373 373
q17 952 499 494 494
q18 7790 6950 6942 6942
q19 1637 1448 1425 1425
q20 542 309 302 302
q21 2856 2433 2433 2433
q22 388 326 337 326
Total cold run time: 111608 ms
Total hot run time: 40043 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5837 5624 5617 5617
q2 357 225 227 225
q3 3245 3213 3187 3187
q4 2194 1998 1969 1969
q5 6111 5963 5976 5963
q6 211 120 126 120
q7 2388 2026 1995 1995
q8 3619 3704 3741 3704
q9 9620 15666 9814 9814
q10 10488 3658 3811 3658
q11 594 461 473 461
q12 1096 678 633 633
q13 17114 3323 3260 3260
q14 329 251 264 251
q15 579 499 519 499
q16 533 488 470 470
q17 5802 1956 1860 1860
q18 9544 9503 11223 9503
q19 13012 1663 1625 1625
q20 3804 1951 1954 1951
q21 14163 5751 5576 5576
q22 2103 578 626 578
Total cold run time: 112743 ms
Total hot run time: 62919 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1892176961
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 774a11e5f5ccc49b7bff6b4a277529b928064250 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.6 seconds inserted 10000000 Rows, about 684K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890903522
<details>
<summary>TPC-H: <b>Total hot run time: 39705 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 598851e65984d583a4da70876e4992e67c0451b5, data reload: false
------ Round 1 ----------------------------------
q1 18072 6647 5725 5725
q2 2039 151 132 132
q3 10884 1149 1240 1149
q4 10522 797 913 797
q5 7817 3143 3189 3143
q6 207 123 126 123
q7 904 497 486 486
q8 9252 1951 2018 1951
q9 7273 6405 6394 6394
q10 8209 3004 3049 3004
q11 416 212 216 212
q12 368 200 202 200
q13 18105 3447 3429 3429
q14 254 220 226 220
q15 569 517 509 509
q16 427 387 375 375
q17 949 509 516 509
q18 7704 7076 6919 6919
q19 1614 1398 1373 1373
q20 552 311 284 284
q21 2889 2442 2465 2442
q22 374 338 329 329
Total cold run time: 109400 ms
Total hot run time: 39705 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5861 5774 5878 5774
q2 351 217 222 217
q3 3350 3171 3168 3168
q4 2212 2033 1945 1945
q5 6081 5975 6041 5975
q6 206 122 125 122
q7 2364 1983 1953 1953
q8 3508 3816 3560 3560
q9 9690 23219 9421 9421
q10 5903 3781 3833 3781
q11 708 457 459 457
q12 1213 664 616 616
q13 17143 3231 3240 3231
q14 381 262 287 262
q15 586 515 513 513
q16 533 488 494 488
q17 9401 2018 1937 1937
q18 9378 9460 15926 9460
q19 13298 1640 1623 1623
q20 4027 1976 1935 1935
q21 13896 5684 5462 5462
q22 1907 594 641 594
Total cold run time: 111997 ms
Total hot run time: 62494 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890908562
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 598851e65984d583a4da70876e4992e67c0451b5 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.7 seconds inserted 10000000 Rows, about 729K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891502405
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1892172208
<details>
<summary>ClickBench: <b>Total hot run time: 31.33 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 774a11e5f5ccc49b7bff6b4a277529b928064250, data reload: false
query1 0.06 0.05 0.05
query2 0.06 0.03 0.02
query3 0.25 0.12 0.12
query4 1.77 0.12 0.12
query5 0.53 0.53 0.52
query6 1.35 0.64 0.64
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.54 0.50 0.51
query10 0.55 0.57 0.55
query11 0.12 0.09 0.09
query12 0.10 0.09 0.09
query13 0.61 0.60 0.60
query14 0.78 0.79 0.77
query15 0.80 0.77 0.77
query16 0.38 0.39 0.38
query17 1.02 1.02 0.99
query18 0.24 0.26 0.27
query19 1.89 1.82 1.78
query20 0.02 0.01 0.01
query21 15.41 0.56 0.56
query22 2.77 2.57 1.76
query23 17.43 0.77 0.74
query24 2.34 1.61 1.36
query25 0.40 0.29 0.11
query26 0.60 0.14 0.15
query27 0.05 0.05 0.05
query28 10.46 0.84 0.81
query29 12.58 3.22 3.21
query30 0.55 0.50 0.50
query31 2.79 0.35 0.35
query32 3.39 0.47 0.49
query33 3.19 3.24 3.25
query34 15.88 4.25 4.20
query35 4.22 4.30 4.21
query36 1.11 1.07 1.09
query37 0.06 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.01 0.02
query40 0.16 0.13 0.13
query41 0.07 0.01 0.02
query42 0.03 0.02 0.02
query43 0.03 0.02 0.02
Total cold run time: 104.68 s
Total hot run time: 31.33 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1897889343
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890907817
<details>
<summary>ClickBench: <b>Total hot run time: 31.07 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 598851e65984d583a4da70876e4992e67c0451b5, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.02 0.03
query3 0.25 0.11 0.12
query4 1.77 0.12 0.11
query5 0.54 0.52 0.52
query6 1.33 0.66 0.63
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.56 0.51 0.51
query10 0.56 0.56 0.58
query11 0.11 0.09 0.08
query12 0.11 0.08 0.09
query13 0.61 0.61 0.61
query14 0.77 0.77 0.80
query15 0.79 0.77 0.77
query16 0.37 0.39 0.38
query17 1.02 1.01 1.01
query18 0.24 0.24 0.26
query19 1.89 1.81 1.81
query20 0.02 0.01 0.01
query21 15.41 0.59 0.57
query22 2.78 2.21 1.74
query23 17.15 0.73 0.73
query24 2.77 1.12 1.82
query25 2.06 0.15 0.15
query26 0.13 0.14 0.13
query27 0.14 0.16 0.14
query28 9.77 0.84 0.79
query29 12.61 3.12 3.10
query30 0.52 0.50 0.49
query31 2.78 0.35 0.36
query32 3.40 0.47 0.49
query33 3.23 3.25 3.21
query34 15.80 4.19 4.22
query35 4.24 4.27 4.31
query36 1.13 1.07 1.08
query37 0.06 0.05 0.06
query38 0.03 0.02 0.02
query39 0.02 0.02 0.01
query40 0.16 0.13 0.13
query41 0.07 0.01 0.02
query42 0.03 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.41 s
Total hot run time: 31.07 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891257503
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1900024829
<details>
<summary>ClickBench: <b>Total hot run time: 29.71 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb880fafdd0764de2e2b4f5e55cc4bc42401dc2a, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.03 0.03
query3 0.22 0.06 0.06
query4 1.66 0.08 0.10
query5 0.54 0.53 0.52
query6 1.27 0.63 0.64
query7 0.02 0.02 0.01
query8 0.04 0.03 0.02
query9 0.56 0.51 0.50
query10 0.56 0.56 0.55
query11 0.12 0.09 0.08
query12 0.11 0.10 0.09
query13 0.60 0.61 0.62
query14 0.78 0.81 0.81
query15 0.78 0.78 0.77
query16 0.37 0.39 0.37
query17 1.01 1.00 1.01
query18 0.25 0.24 0.24
query19 1.82 1.82 1.69
query20 0.01 0.01 0.01
query21 15.42 0.58 0.57
query22 2.77 2.99 1.26
query23 17.46 0.77 0.69
query24 2.42 1.31 0.46
query25 0.42 0.27 0.20
query26 0.34 0.14 0.13
query27 0.06 0.07 0.05
query28 11.64 0.78 0.75
query29 12.50 3.25 3.18
query30 0.53 0.48 0.48
query31 2.78 0.34 0.35
query32 3.38 0.48 0.48
query33 3.22 3.25 3.20
query34 15.72 4.37 4.23
query35 4.33 4.27 4.30
query36 1.11 1.07 1.08
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.02 0.01
query40 0.15 0.14 0.13
query41 0.08 0.02 0.02
query42 0.03 0.01 0.01
query43 0.02 0.01 0.02
Total cold run time: 105.32 s
Total hot run time: 29.71 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #29953:
URL: https://github.com/apache/doris/pull/29953
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890896718
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891239832
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891254692
<details>
<summary>TPC-H: <b>Total hot run time: 39887 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1000e47f26f772a5591f1a215e7317d2d6e9803f, data reload: false
------ Round 1 ----------------------------------
q1 17994 6106 6053 6053
q2 2045 144 140 140
q3 10879 1124 1203 1124
q4 10534 767 843 767
q5 7842 3197 3124 3124
q6 211 130 128 128
q7 908 505 481 481
q8 9252 2001 2008 2001
q9 7316 6431 6394 6394
q10 8244 3068 3100 3068
q11 416 206 211 206
q12 365 202 191 191
q13 18124 3420 3419 3419
q14 251 217 223 217
q15 566 517 519 517
q16 427 371 401 371
q17 941 539 483 483
q18 7562 6821 6826 6821
q19 2571 1341 1383 1341
q20 546 321 286 286
q21 2826 2503 2442 2442
q22 361 313 328 313
Total cold run time: 110181 ms
Total hot run time: 39887 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5906 6070 5756 5756
q2 354 219 221 219
q3 3350 3090 3188 3090
q4 2229 2037 1976 1976
q5 6103 5936 5940 5936
q6 208 125 125 125
q7 2368 1967 1994 1967
q8 3656 3817 3735 3735
q9 9690 24448 9279 9279
q10 10357 3823 3900 3823
q11 714 441 475 441
q12 1465 650 650 650
q13 17365 3269 3227 3227
q14 372 257 264 257
q15 574 507 508 507
q16 551 504 483 483
q17 9163 2032 2108 2032
q18 9478 9476 12320 9476
q19 12969 1611 1639 1611
q20 3990 1945 1933 1933
q21 13955 5691 5920 5691
q22 1250 613 621 613
Total cold run time: 116067 ms
Total hot run time: 62827 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891552525
<details>
<summary>TPC-H: <b>Total hot run time: 39460 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ddacbde5f3c406be66a3eb10ab3ee2a45cf10edd, data reload: false
------ Round 1 ----------------------------------
q1 17976 6297 5722 5722
q2 2036 146 133 133
q3 10897 1155 1213 1155
q4 10532 774 866 774
q5 7819 3120 3189 3120
q6 206 125 123 123
q7 905 504 484 484
q8 9297 1969 2004 1969
q9 7273 6410 6381 6381
q10 8228 3053 3050 3050
q11 425 212 207 207
q12 359 196 200 196
q13 18107 3401 3398 3398
q14 250 225 214 214
q15 559 522 505 505
q16 442 373 396 373
q17 933 545 535 535
q18 7706 7021 6820 6820
q19 1579 1285 1420 1285
q20 576 309 293 293
q21 2885 2411 2467 2411
q22 371 312 312 312
Total cold run time: 109361 ms
Total hot run time: 39460 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5753 5548 5628 5548
q2 349 222 221 221
q3 3341 3202 3119 3119
q4 2196 2055 1946 1946
q5 6074 5870 6089 5870
q6 206 122 120 120
q7 2438 1943 1961 1943
q8 3545 3909 3721 3721
q9 9585 20388 9439 9439
q10 8373 3743 3840 3743
q11 586 444 461 444
q12 1181 643 657 643
q13 17120 3320 3283 3283
q14 288 256 263 256
q15 575 513 508 508
q16 530 485 475 475
q17 2311 2064 2011 2011
q18 9553 9211 9188 9188
q19 16025 1661 1603 1603
q20 4544 1950 1944 1944
q21 13949 5765 5634 5634
q22 1013 606 606 606
Total cold run time: 109535 ms
Total hot run time: 62265 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894064471
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894056706
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1897830963
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1900000281
<details>
<summary>TPC-H: <b>Total hot run time: 38840 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb880fafdd0764de2e2b4f5e55cc4bc42401dc2a, data reload: false
------ Round 1 ----------------------------------
q1 17751 5660 5376 5376
q2 2530 141 142 141
q3 11404 1175 1161 1161
q4 10860 733 776 733
q5 7715 3150 3143 3143
q6 203 120 121 120
q7 873 489 505 489
q8 9233 1917 1960 1917
q9 7251 6358 6349 6349
q10 8200 3047 3038 3038
q11 426 205 218 205
q12 361 194 190 190
q13 17993 3378 3337 3337
q14 243 218 214 214
q15 549 512 509 509
q16 423 362 387 362
q17 939 563 501 501
q18 7569 6999 6769 6769
q19 1556 1297 1393 1297
q20 554 316 331 316
q21 2737 2468 2356 2356
q22 353 325 317 317
Total cold run time: 109723 ms
Total hot run time: 38840 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5423 5332 5363 5332
q2 328 219 210 210
q3 3344 3246 3248 3246
q4 2083 2034 2066 2034
q5 6023 5933 5859 5859
q6 196 118 114 114
q7 2327 1803 1882 1803
q8 3215 3386 3393 3386
q9 8955 8672 8797 8672
q10 3926 3758 3857 3758
q11 572 459 452 452
q12 788 624 612 612
q13 16930 3136 3169 3136
q14 292 257 279 257
q15 550 496 512 496
q16 513 471 491 471
q17 1879 1845 1834 1834
q18 9511 19366 9597 9597
q19 24359 1579 1516 1516
q20 4585 1934 1916 1916
q21 14494 5395 5391 5391
q22 999 564 536 536
Total cold run time: 111292 ms
Total hot run time: 60628 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1892114157
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894159326
<details>
<summary>TPC-DS: <b>Total hot run time: 176974 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit af4788261e9536e2128a00befe9ff4be6c1114ff, data reload: false
query1 935 340 330 330
query2 6564 1879 1933 1879
query3 6706 208 209 208
query4 31176 22148 22340 22148
query5 6910 545 527 527
query6 255 173 170 170
query7 4596 274 263 263
query8 231 180 195 180
query9 8385 2541 2530 2530
query10 415 228 242 228
query11 16933 15512 15511 15511
query12 119 68 70 68
query13 1678 381 376 376
query14 10534 6836 6824 6824
query15 211 178 180 178
query16 4608 236 231 231
query17 980 484 484 484
query18 1788 261 253 253
query19 176 141 136 136
query20 74 70 62 62
query21 184 125 129 125
query22 4998 4688 4752 4688
query23 31930 30626 30640 30626
query24 7429 2827 2853 2827
query25 407 323 319 319
query26 718 141 142 141
query27 2298 280 272 272
query28 4030 1840 1823 1823
query29 919 638 625 625
query30 195 138 140 138
query31 919 742 744 742
query32 61 55 51 51
query33 421 216 214 214
query34 815 451 454 451
query35 869 796 762 762
query36 1344 1273 1208 1208
query37 93 59 58 58
query38 3367 3255 3214 3214
query39 1327 1240 1258 1240
query40 195 92 83 83
query41 38 36 34 34
query42 86 84 90 84
query43 551 500 529 500
query44 1020 677 700 677
query45 197 184 175 175
query46 1046 649 655 649
query47 1710 1619 1533 1533
query48 414 328 315 315
query49 628 297 287 287
query50 687 309 325 309
query51 5304 5183 5161 5161
query52 93 89 76 76
query53 328 261 254 254
query54 720 439 450 439
query55 82 79 74 74
query56 176 163 165 163
query57 990 964 932 932
query58 188 156 163 156
query59 2971 2758 2790 2758
query60 212 189 188 188
query61 87 84 86 84
query62 475 397 409 397
query63 287 273 257 257
query64 2626 1833 1795 1795
query65 3334 3254 3263 3254
query66 1043 323 312 312
query67 15763 14933 15133 14933
query68 12291 509 506 506
query69 639 297 296 296
query70 1698 1545 1509 1509
query71 503 210 213 210
query72 5127 2847 2838 2838
query73 3191 315 314 314
query74 6936 6380 6426 6380
query75 5315 2271 2329 2271
query76 5995 1030 1048 1030
query77 681 243 239 239
query78 9355 8681 8684 8681
query79 1055 503 503 503
query80 602 317 311 311
query81 474 220 204 204
query82 204 87 77 77
query83 140 120 122 120
query84 274 68 73 68
query85 1080 328 324 324
query86 389 385 378 378
query87 3563 3343 3387 3343
query88 2901 2265 2258 2258
query89 445 359 365 359
query90 2222 196 191 191
query91 157 128 123 123
query92 57 45 42 42
query93 1853 410 402 402
query94 1496 160 158 158
query95 508 458 454 454
query96 630 339 331 331
query97 4306 4144 4185 4144
query98 198 192 190 190
query99 1031 742 731 731
Total cold run time: 282165 ms
Total hot run time: 176974 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1894173814
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit af4788261e9536e2128a00befe9ff4be6c1114ff with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.6 seconds inserted 10000000 Rows, about 684K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and catch and record exception for every materialization context [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1890889789
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure to catch and record exception for every materialization context [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1903158057
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891261368
<details>
<summary>TPC-DS: <b>Total hot run time: 178740 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1000e47f26f772a5591f1a215e7317d2d6e9803f, data reload: false
query1 924 329 326 326
query2 6748 1899 1903 1899
query3 6729 213 212 212
query4 37020 22209 22408 22209
query5 6930 594 548 548
query6 260 180 182 180
query7 4611 276 281 276
query8 228 213 185 185
query9 9184 2730 2810 2730
query10 458 240 241 240
query11 16337 15382 15170 15170
query12 121 74 70 70
query13 1702 386 372 372
query14 11231 7526 7359 7359
query15 222 183 188 183
query16 4615 246 244 244
query17 1018 494 487 487
query18 1807 262 255 255
query19 184 150 151 150
query20 79 80 75 75
query21 202 135 126 126
query22 4907 4849 4657 4657
query23 31869 31159 31045 31045
query24 11444 2825 2821 2821
query25 569 329 320 320
query26 1664 153 156 153
query27 3232 284 282 282
query28 7312 1870 1860 1860
query29 1731 702 682 682
query30 283 139 142 139
query31 947 772 776 772
query32 77 63 55 55
query33 702 246 244 244
query34 1142 472 465 465
query35 906 789 781 781
query36 1458 1298 1228 1228
query37 99 66 72 66
query38 3423 3315 3230 3230
query39 1331 1279 1252 1252
query40 340 99 95 95
query41 39 34 35 34
query42 101 98 95 95
query43 568 537 535 535
query44 1094 710 718 710
query45 202 190 187 187
query46 1078 630 641 630
query47 1677 1601 1621 1601
query48 427 309 314 309
query49 1155 314 312 312
query50 701 324 319 319
query51 5317 5279 5242 5242
query52 106 90 93 90
query53 358 283 278 278
query54 909 449 464 449
query55 100 91 85 85
query56 209 201 194 194
query57 1019 979 940 940
query58 217 186 190 186
query59 3062 2799 2895 2799
query60 243 226 232 226
query61 84 80 84 80
query62 692 401 403 401
query63 319 285 297 285
query64 5472 1679 1650 1650
query65 3387 3283 3299 3283
query66 1283 323 328 323
query67 15650 14983 15150 14983
query68 12893 503 504 503
query69 640 339 333 333
query70 1913 1560 1534 1534
query71 548 239 244 239
query72 5192 2843 2850 2843
query73 3251 325 314 314
query74 6986 6376 6463 6376
query75 5171 2360 2315 2315
query76 6300 972 1077 972
query77 715 250 264 250
query78 9362 8860 8521 8521
query79 2460 512 500 500
query80 564 341 328 328
query81 447 202 204 202
query82 213 94 91 91
query83 144 125 118 118
query84 274 74 73 73
query85 1072 323 326 323
query86 429 398 425 398
query87 3596 3355 3388 3355
query88 3419 2247 2225 2225
query89 461 418 407 407
query90 2027 207 194 194
query91 153 123 127 123
query92 60 56 55 55
query93 1629 428 430 428
query94 1212 177 174 174
query95 530 474 459 459
query96 614 321 314 314
query97 4322 4183 4160 4160
query98 216 187 199 187
query99 1060 757 760 757
Total cold run time: 307388 ms
Total hot run time: 178740 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891258664
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.8 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
storage size: 17183916174 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891569985
<details>
<summary>TPC-DS: <b>Total hot run time: 179571 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ddacbde5f3c406be66a3eb10ab3ee2a45cf10edd, data reload: false
query1 959 340 336 336
query2 6693 1859 1828 1828
query3 6728 211 206 206
query4 31977 22223 22214 22214
query5 6943 585 545 545
query6 262 185 179 179
query7 4617 283 267 267
query8 243 195 197 195
query9 9099 2803 2754 2754
query10 633 242 241 241
query11 16448 15519 15304 15304
query12 136 77 73 73
query13 1704 382 382 382
query14 11281 7478 7668 7478
query15 227 203 196 196
query16 4708 268 250 250
query17 1162 524 513 513
query18 1835 287 272 272
query19 195 157 163 157
query20 87 87 84 84
query21 200 141 136 136
query22 5437 5161 5176 5161
query23 32231 31054 30993 30993
query24 11341 2888 2893 2888
query25 564 324 321 321
query26 1023 151 152 151
query27 3126 279 278 278
query28 6699 1891 1851 1851
query29 1169 679 664 664
query30 280 137 142 137
query31 917 762 771 762
query32 79 58 61 58
query33 712 230 245 230
query34 1148 459 452 452
query35 882 777 776 776
query36 1459 1279 1290 1279
query37 102 71 70 70
query38 3401 3275 3277 3275
query39 1334 1270 1275 1270
query40 220 91 96 91
query41 40 35 34 34
query42 104 95 90 90
query43 586 517 512 512
query44 1097 718 716 716
query45 201 187 185 185
query46 1066 652 630 630
query47 1674 1605 1536 1536
query48 414 333 312 312
query49 1142 321 321 321
query50 715 321 315 315
query51 5304 5216 5237 5216
query52 109 84 95 84
query53 360 306 288 288
query54 896 455 452 452
query55 95 87 94 87
query56 212 194 189 189
query57 1001 978 943 943
query58 206 197 190 190
query59 3067 2689 2801 2689
query60 232 234 211 211
query61 84 82 83 82
query62 670 427 413 413
query63 309 286 281 281
query64 4904 1681 1717 1681
query65 3350 3284 3279 3279
query66 1249 326 322 322
query67 15567 15068 14988 14988
query68 11782 512 519 512
query69 637 332 335 332
query70 1791 1653 1574 1574
query71 576 238 258 238
query72 5136 2883 2852 2852
query73 2384 314 322 314
query74 6949 6426 6546 6426
query75 4945 2315 2315 2315
query76 6271 995 989 989
query77 684 266 286 266
query78 9466 8881 8517 8517
query79 1019 528 511 511
query80 555 336 342 336
query81 471 207 201 201
query82 210 93 93 93
query83 139 129 124 124
query84 282 72 77 72
query85 1043 348 318 318
query86 399 375 383 375
query87 3534 3409 3421 3409
query88 3298 2235 2231 2231
query89 459 399 399 399
query90 2017 196 194 194
query91 154 122 127 122
query92 59 62 52 52
query93 1879 431 413 413
query94 1236 182 174 174
query95 522 477 464 464
query96 616 310 321 310
query97 4317 4142 4196 4142
query98 215 194 190 190
query99 1029 743 765 743
Total cold run time: 296989 ms
Total hot run time: 179571 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](nereids) Make sure instance when change params of StructInfo,Predicates and record catched exception for every materialization context [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29953:
URL: https://github.com/apache/doris/pull/29953#issuecomment-1891760697
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org