You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "XieJiann (via GitHub)" <gi...@apache.org> on 2023/12/13 09:33:15 UTC

[PR] [feature](Nereids): put or expansion rule in cost base rewrite job [doris]

XieJiann opened a new pull request, #28348:
URL: https://github.com/apache/doris/pull/28348

   ## Proposed changes
   put or expansion rule in cost base rewrite job
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854261607

   run builall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859440888

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859457660

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit ae77f54a9adf769fc962fd3afca689667b9e8cc5, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4683	4481	4477	4477
   q2	365	151	156	151
   q3	1460	1257	1280	1257
   q4	1116	885	849	849
   q5	3147	3128	3165	3128
   q6	245	129	130	129
   q7	987	491	486	486
   q8	2231	2201	2209	2201
   q9	6688	6680	6842	6680
   q10	3221	3288	3246	3246
   q11	327	202	200	200
   q12	347	214	207	207
   q13	4567	3842	3802	3802
   q14	243	212	210	210
   q15	568	526	518	518
   q16	445	390	381	381
   q17	1014	583	511	511
   q18	7182	6932	7119	6932
   q19	1534	1446	1388	1388
   q20	562	317	304	304
   q21	3091	2630	2696	2630
   q22	342	283	282	282
   Total cold run time: 44365 ms
   Total hot run time: 39969 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4396	4390	4370	4370
   q2	270	166	173	166
   q3	3537	3533	3529	3529
   q4	2383	2367	2364	2364
   q5	5737	5731	5722	5722
   q6	241	122	123	122
   q7	2372	1899	1860	1860
   q8	3526	3540	3519	3519
   q9	8948	8969	8933	8933
   q10	3916	4018	3983	3983
   q11	496	385	392	385
   q12	767	606	602	602
   q13	4274	3579	3594	3579
   q14	290	257	258	257
   q15	574	525	532	525
   q16	501	468	463	463
   q17	1886	1869	1854	1854
   q18	8756	8145	8258	8145
   q19	1724	1724	1763	1724
   q20	2262	1949	1936	1936
   q21	6517	6203	6181	6181
   q22	509	420	413	413
   Total cold run time: 63882 ms
   Total hot run time: 60632 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427496409


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/OrExpansion.java:
##########
@@ -72,7 +72,7 @@ public class OrExpansion extends OneExplorationRuleFactory {
 
     @Override
     public Rule build() {
-        return logicalJoin().when(JoinUtils::shouldNestedLoopJoin)
+        return logicalJoin(any(), any()).when(JoinUtils::shouldNestedLoopJoin)

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1853784571

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.69 seconds
    stream load tsv:          591 seconds loaded 74807831229 Bytes, about 120 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
    insert into select:          29.6 seconds inserted 10000000 Rows, about 337K ops/s
    storage size: 17224213461 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855188919

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.54 seconds
    stream load tsv:          592 seconds loaded 74807831229 Bytes, about 120 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
    storage size: 17221251426 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855086101

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit d86bae96b450d5b9ada18e1c1dfa03b2915b12c6, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4779	4468	4460	4460
   q2	371	152	159	152
   q3	1491	1233	1237	1233
   q4	1119	932	864	864
   q5	3215	3212	3199	3199
   q6	246	128	128	128
   q7	1002	496	480	480
   q8	2224	2225	2216	2216
   q9	6760	6703	6725	6703
   q10	3273	3283	3264	3264
   q11	324	211	198	198
   q12	362	206	216	206
   q13	4576	3841	3775	3775
   q14	243	212	213	212
   q15	576	531	540	531
   q16	442	382	391	382
   q17	1027	641	607	607
   q18	7076	6690	6681	6681
   q19	1568	1510	1253	1253
   q20	524	306	283	283
   q21	3073	2665	2671	2665
   q22	356	278	285	278
   Total cold run time: 44627 ms
   Total hot run time: 39770 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4422	4392	4423	4392
   q2	269	164	175	164
   q3	3523	3509	3499	3499
   q4	2422	2404	2407	2404
   q5	5746	5760	5762	5760
   q6	237	121	123	121
   q7	2356	1830	1845	1830
   q8	3540	3546	3550	3546
   q9	9108	9068	9102	9068
   q10	3920	4009	4027	4009
   q11	514	381	390	381
   q12	771	607	608	607
   q13	4352	3551	3547	3547
   q14	288	253	252	252
   q15	578	524	528	524
   q16	517	481	474	474
   q17	1917	1867	1864	1864
   q18	8542	8176	7977	7977
   q19	1768	1754	1766	1754
   q20	2246	1945	1954	1945
   q21	6568	6189	6154	6154
   q22	504	422	419	419
   Total cold run time: 64108 ms
   Total hot run time: 60691 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857187402

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859641887

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427480947


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
                 topic("rewrite cte sub-tree",
                         custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
                 ),
+                topic("or expansion",

Review Comment:
   Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plam



##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
                 topic("rewrite cte sub-tree",
                         custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
                 ),
+                topic("or expansion",

Review Comment:
   Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plan



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427480947


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
                 topic("rewrite cte sub-tree",
                         custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
                 ),
+                topic("or expansion",

Review Comment:
   Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plan in the rewritten CTE children



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857516949

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1853718627

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1426502206


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
                 topic("rewrite cte sub-tree",
                         custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
                 ),
+                topic("or expansion",

Review Comment:
   what is the reason to put it here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "jackwener (via GitHub)" <gi...@apache.org>.
jackwener merged PR #28348:
URL: https://github.com/apache/doris/pull/28348


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854261760

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854400195

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.77 seconds
    stream load tsv:          590 seconds loaded 74807831229 Bytes, about 120 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
    insert into select:          28.2 seconds inserted 10000000 Rows, about 354K ops/s
    storage size: 17218137664 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854375464

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 89ce4dec12b6e2d24ca57a5af2aaab35f77ee1f5, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	50237	31876	31858	31858
   q2	374	135	148	135
   q3	1488	1279	1198	1198
   q4	1116	912	905	905
   q5	3165	3195	3210	3195
   q6	260	138	152	138
   q7	989	500	480	480
   q8	2140	2189	2179	2179
   q9	6959	6843	6886	6843
   q10	3246	3299	3308	3299
   q11	344	223	227	223
   q12	350	208	210	208
   q13	4606	4371	3842	3842
   q14	424	317	317	317
   q15	788	1314	1347	1314
   q16	438	392	388	388
   q17	1022	615	521	521
   q18	7381	7012	6894	6894
   q19	1548	1452	1420	1420
   q20	1810	326	400	326
   q21	3076	2610	2650	2610
   q22	347	282	281	281
   Total cold run time: 92108 ms
   Total hot run time: 68574 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	30837	31869	31830	31830
   q2	279	165	170	165
   q3	3607	3627	3598	3598
   q4	2384	2372	2368	2368
   q5	5741	5750	5755	5750
   q6	240	130	135	130
   q7	3033	3404	3373	3373
   q8	3540	3530	3544	3530
   q9	9175	9146	9187	9146
   q10	3952	4052	4091	4052
   q11	807	917	1375	917
   q12	784	626	604	604
   q13	4302	3541	3554	3541
   q14	521	442	434	434
   q15	1108	1346	1344	1344
   q16	496	452	457	452
   q17	1890	1877	1831	1831
   q18	8717	8145	8167	8145
   q19	1758	1797	1771	1771
   q20	2892	2622	2704	2622
   q21	6511	6164	6145	6145
   q22	508	419	415	415
   Total cold run time: 93082 ms
   Total hot run time: 92163 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857646461

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859483058

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.3 seconds
    stream load tsv:          585 seconds loaded 74807831229 Bytes, about 121 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.1 seconds inserted 10000000 Rows, about 355K ops/s
    storage size: 17219678315 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855178907

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit d1110b9960b9403134fb2ea7e3631073c3161fa9, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4723	4478	4492	4478
   q2	378	157	163	157
   q3	1463	1266	1238	1238
   q4	1135	910	859	859
   q5	3197	3210	3202	3202
   q6	246	125	125	125
   q7	1027	489	477	477
   q8	2181	2222	2195	2195
   q9	6704	6656	6622	6622
   q10	3232	3280	3268	3268
   q11	320	197	201	197
   q12	348	220	203	203
   q13	4539	3831	3752	3752
   q14	242	220	213	213
   q15	577	527	517	517
   q16	444	386	386	386
   q17	1012	574	567	567
   q18	7091	6704	6896	6704
   q19	1536	1455	1362	1362
   q20	550	327	308	308
   q21	3044	2654	2665	2654
   q22	348	278	286	278
   Total cold run time: 44337 ms
   Total hot run time: 39762 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4407	4419	4385	4385
   q2	268	162	169	162
   q3	3507	3493	3491	3491
   q4	2403	2375	2399	2375
   q5	5754	5741	5764	5741
   q6	233	123	122	122
   q7	2364	1834	1842	1834
   q8	3536	3520	3548	3520
   q9	9039	9001	8993	8993
   q10	3930	4004	3992	3992
   q11	505	375	380	375
   q12	758	590	588	588
   q13	4272	3590	3582	3582
   q14	294	261	260	260
   q15	568	515	514	514
   q16	528	476	458	458
   q17	1895	1860	1852	1852
   q18	8491	8095	8166	8095
   q19	1749	1750	1739	1739
   q20	2241	1945	1935	1935
   q21	6495	6116	6142	6116
   q22	508	426	422	422
   Total cold run time: 63745 ms
   Total hot run time: 60551 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859641821

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855056492

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857697814

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.52 seconds
    stream load tsv:          585 seconds loaded 74807831229 Bytes, about 121 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          29.0 seconds inserted 10000000 Rows, about 344K ops/s
    storage size: 17219582542 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855057768

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859478716

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit ae77f54a9adf769fc962fd3afca689667b9e8cc5, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4723	4467	4528	4467
   q2	367	149	158	149
   q3	1461	1215	1246	1215
   q4	1109	939	881	881
   q5	3165	3168	3150	3150
   q6	250	127	129	127
   q7	1016	486	500	486
   q8	2198	2196	2180	2180
   q9	6688	6705	6680	6680
   q10	3227	3278	3260	3260
   q11	328	211	204	204
   q12	352	213	208	208
   q13	4559	3770	3818	3770
   q14	244	215	218	215
   q15	576	527	521	521
   q16	443	389	385	385
   q17	1022	628	571	571
   q18	7210	6888	6983	6888
   q19	1507	1452	1386	1386
   q20	547	291	314	291
   q21	3096	2607	2666	2607
   q22	353	278	283	278
   Total cold run time: 44441 ms
   Total hot run time: 39919 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4431	4412	4394	4394
   q2	266	164	168	164
   q3	3541	3525	3525	3525
   q4	2390	2371	2370	2370
   q5	5760	5742	5749	5742
   q6	243	123	121	121
   q7	2394	1855	1887	1855
   q8	3524	3523	3522	3522
   q9	9023	8934	8996	8934
   q10	3938	3979	4005	3979
   q11	499	387	372	372
   q12	777	604	587	587
   q13	4290	3581	3535	3535
   q14	288	256	268	256
   q15	572	524	515	515
   q16	506	464	475	464
   q17	1870	1848	1864	1848
   q18	8668	8316	8373	8316
   q19	1746	1744	1744	1744
   q20	2278	1962	1947	1947
   q21	6494	6166	6157	6157
   q22	509	425	426	425
   Total cold run time: 64007 ms
   Total hot run time: 60772 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855141570

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]

Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1426494587


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/OrExpansion.java:
##########
@@ -72,7 +72,7 @@ public class OrExpansion extends OneExplorationRuleFactory {
 
     @Override
     public Rule build() {
-        return logicalJoin().when(JoinUtils::shouldNestedLoopJoin)
+        return logicalJoin(any(), any()).when(JoinUtils::shouldNestedLoopJoin)

Review Comment:
   better to move this file from exploration to rewriter dir?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org