You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/09 11:22:11 UTC

[PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Gabriel39 opened a new pull request, #29738:
URL: https://github.com/apache/doris/pull/29738

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882890479

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882953344

   TeamCity be ut coverage result:
    Function Coverage: 36.66% (8645/23581) 
    Line Coverage: 28.72% (70436/245273)
    Region Coverage: 27.64% (36387/131637)
    Branch Coverage: 24.33% (18600/76440)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/24ca4e12b9505098cb707d7c15b41a53756c0b8f_24ca4e12b9505098cb707d7c15b41a53756c0b8f/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882943246

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 40330 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	18278	5579	5551	5551
   q2	2037	163	146	146
   q3	10777	1208	1211	1208
   q4	10337	845	870	845
   q5	7921	3066	3023	3023
   q6	210	132	137	132
   q7	958	528	518	518
   q8	9422	2061	2057	2057
   q9	6923	6629	6594	6594
   q10	8393	3058	3061	3058
   q11	448	215	217	215
   q12	396	228	232	228
   q13	18289	3467	3463	3463
   q14	243	214	217	214
   q15	560	532	514	514
   q16	471	418	424	418
   q17	956	791	664	664
   q18	7607	6953	6887	6887
   q19	1578	1540	1524	1524
   q20	733	338	329	329
   q21	2777	2483	2411	2411
   q22	384	348	331	331
   Total cold run time: 109698 ms
   Total hot run time: 40330 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5515	5500	5524	5500
   q2	357	258	261	258
   q3	3424	3352	3319	3319
   q4	2126	2071	2107	2071
   q5	5888	5846	5912	5846
   q6	208	124	129	124
   q7	2350	1883	1936	1883
   q8	3542	3622	3556	3556
   q9	8981	8960	8919	8919
   q10	3890	3920	3918	3918
   q11	600	503	506	503
   q12	817	641	647	641
   q13	3950	3260	3238	3238
   q14	299	261	270	261
   q15	567	518	513	513
   q16	550	482	491	482
   q17	2458	2388	2387	2387
   q18	9088	8817	8859	8817
   q19	1594	1634	1636	1634
   q20	2260	1987	1950	1950
   q21	5791	5488	5249	5249
   q22	602	539	520	520
   Total cold run time: 64857 ms
   Total hot run time: 61589 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882961962

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.26 seconds
    stream load tsv:          580 seconds loaded 74807831229 Bytes, about 123 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          21.8 seconds inserted 10000000 Rows, about 458K ops/s
    storage size: 17184032047 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882968994

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 31.32 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
   
   query1	0.11	0.07	0.06
   query2	0.07	0.02	0.02
   query3	0.32	0.21	0.20
   query4	1.71	0.19	0.19
   query5	0.54	0.53	0.54
   query6	1.58	0.68	0.70
   query7	0.02	0.01	0.01
   query8	0.04	0.02	0.03
   query9	0.62	0.57	0.58
   query10	0.63	0.63	0.64
   query11	0.17	0.13	0.11
   query12	0.15	0.13	0.11
   query13	0.61	0.61	0.61
   query14	0.80	0.79	0.79
   query15	0.84	0.82	0.82
   query16	0.35	0.34	0.35
   query17	0.97	0.95	1.01
   query18	0.27	0.26	0.27
   query19	1.85	1.80	1.77
   query20	0.01	0.01	0.01
   query21	15.42	0.54	0.54
   query22	2.93	3.25	0.73
   query23	17.12	1.15	1.06
   query24	10.03	0.34	0.98
   query25	2.44	0.18	0.18
   query26	0.16	0.15	0.16
   query27	0.17	0.17	0.17
   query28	5.32	0.94	0.87
   query29	13.13	3.53	3.53
   query30	0.58	0.50	0.49
   query31	2.92	0.38	0.40
   query32	3.27	0.49	0.50
   query33	3.29	3.27	3.29
   query34	16.19	4.73	4.66
   query35	4.60	4.76	4.45
   query36	1.17	1.09	1.11
   query37	0.07	0.04	0.04
   query38	0.03	0.03	0.02
   query39	0.02	0.01	0.02
   query40	0.16	0.13	0.14
   query41	0.07	0.01	0.02
   query42	0.02	0.01	0.01
   query43	0.03	0.02	0.02
   Total cold run time: 110.8 s
   Total hot run time: 31.32 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882973575

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f with default session variables
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
    stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:       17.4 seconds inserted 10000000 Rows, about 574K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882901097

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29738:
URL: https://github.com/apache/doris/pull/29738


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882894246

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882894282

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882960724

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 186608 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
   
   run tpcds-sf100 query with default conf and session variables
   query1	955	351	339	339
   query2	6839	2054	1990	1990
   query3	6726	232	233	232
   query4	26494	22594	22487	22487
   query5	6164	628	583	583
   query6	334	220	224	220
   query7	4669	304	309	304
   query8	262	243	224	224
   query9	9720	3305	3340	3305
   query10	589	355	307	307
   query11	16411	15670	15623	15623
   query12	173	116	112	112
   query13	1681	333	336	333
   query14	13966	7509	7668	7509
   query15	295	246	240	240
   query16	6384	362	357	357
   query17	1684	563	530	530
   query18	1956	343	332	332
   query19	346	178	161	161
   query20	134	118	119	118
   query21	202	103	99	99
   query22	5009	4710	4801	4710
   query23	32187	31206	31575	31206
   query24	12036	3069	3017	3017
   query25	620	400	379	379
   query26	1768	177	182	177
   query27	2756	323	304	304
   query28	7157	2162	2135	2135
   query29	2029	458	434	434
   query30	287	149	151	149
   query31	1053	823	847	823
   query32	142	91	91	91
   query33	837	350	360	350
   query34	887	490	508	490
   query35	1040	847	847	847
   query36	1368	1357	1391	1357
   query37	228	130	107	107
   query38	3516	3378	3333	3333
   query39	1384	1298	1314	1298
   query40	310	92	94	92
   query41	41	42	40	40
   query42	119	119	116	116
   query43	620	563	583	563
   query44	1307	826	852	826
   query45	216	199	212	199
   query46	1093	714	711	711
   query47	1773	1740	1729	1729
   query48	388	289	290	289
   query49	1215	326	322	322
   query50	799	342	369	342
   query51	5494	5341	5339	5339
   query52	125	115	112	112
   query53	236	182	176	176
   query54	1395	716	711	711
   query55	111	101	102	101
   query56	290	285	269	269
   query57	1137	1041	1013	1013
   query58	347	327	338	327
   query59	3331	3142	3120	3120
   query60	345	332	300	300
   query61	152	154	150	150
   query62	590	494	502	494
   query63	213	203	205	203
   query64	6119	1753	1768	1753
   query65	3436	3377	3375	3375
   query66	1349	427	393	393
   query67	15832	15711	15481	15481
   query68	12884	556	545	545
   query69	616	373	367	367
   query70	1818	1800	1738	1738
   query71	618	306	271	271
   query72	5482	3436	3551	3436
   query73	3024	336	346	336
   query74	7192	6586	6617	6586
   query75	5330	2313	2333	2313
   query76	6368	1049	1245	1049
   query77	763	365	331	331
   query78	9103	8849	8686	8686
   query79	4590	548	547	547
   query80	703	360	362	360
   query81	467	220	232	220
   query82	241	141	140	140
   query83	178	147	147	147
   query84	254	58	63	58
   query85	968	280	284	280
   query86	454	469	471	469
   query87	3630	3496	3448	3448
   query88	3644	2550	2537	2537
   query89	415	321	327	321
   query90	1943	273	284	273
   query91	135	95	100	95
   query92	116	91	85	85
   query93	2025	496	426	426
   query94	863	237	254	237
   query95	603	530	519	519
   query96	666	337	337	337
   query97	4420	4287	4219	4219
   query98	239	215	212	212
   query99	1191	898	862	862
   Total cold run time: 309614 ms
   Total hot run time: 186608 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org