You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/11 08:22:40 UTC

[PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Gabriel39 opened a new pull request, #29860:
URL: https://github.com/apache/doris/pull/29860

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886666964

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 179537 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
   
   query1	941	333	323	323
   query2	6712	1922	1827	1827
   query3	6699	212	210	210
   query4	25894	22247	22123	22123
   query5	5163	520	541	520
   query6	272	194	185	185
   query7	4624	284	281	281
   query8	247	208	200	200
   query9	9224	2754	2925	2754
   query10	457	250	241	241
   query11	16151	15376	15326	15326
   query12	140	78	72	72
   query13	1703	399	390	390
   query14	11737	7355	7288	7288
   query15	238	191	198	191
   query16	6479	261	253	253
   query17	1879	481	473	473
   query18	1943	266	278	266
   query19	279	153	156	153
   query20	84	79	76	76
   query21	185	137	126	126
   query22	5061	4719	4714	4714
   query23	32226	30935	30811	30811
   query24	10957	2890	2881	2881
   query25	559	337	329	329
   query26	1346	150	160	150
   query27	2826	281	281	281
   query28	6949	1889	1872	1872
   query29	1160	678	655	655
   query30	273	138	139	138
   query31	980	777	788	777
   query32	98	63	59	59
   query33	704	247	263	247
   query34	972	484	468	468
   query35	910	765	780	765
   query36	1395	1222	1244	1222
   query37	98	78	73	73
   query38	3412	3282	3252	3252
   query39	1346	1280	1267	1267
   query40	215	100	100	100
   query41	37	36	36	36
   query42	107	98	99	98
   query43	534	517	517	517
   query44	1095	715	722	715
   query45	212	188	182	182
   query46	1061	665	676	665
   query47	1634	1636	1604	1604
   query48	403	323	340	323
   query49	1151	334	323	323
   query50	706	317	328	317
   query51	5299	5211	5233	5211
   query52	102	90	94	90
   query53	366	300	293	293
   query54	2091	1645	1626	1626
   query55	106	85	90	85
   query56	215	201	203	201
   query57	1033	965	946	946
   query58	216	201	207	201
   query59	2754	2563	2461	2461
   query60	233	218	232	218
   query61	86	81	80	80
   query62	605	419	402	402
   query63	299	297	294	294
   query64	4944	1676	1742	1676
   query65	3357	3258	3285	3258
   query66	1049	339	318	318
   query67	15604	15281	15051	15051
   query68	13021	553	525	525
   query69	670	375	392	375
   query70	1685	1499	1529	1499
   query71	506	247	243	243
   query72	5091	2858	2814	2814
   query73	3046	311	317	311
   query74	6923	6406	6490	6406
   query75	5376	2364	2345	2345
   query76	6380	900	1141	900
   query77	697	278	266	266
   query78	9993	8667	8465	8465
   query79	1723	525	504	504
   query80	579	345	345	345
   query81	472	207	213	207
   query82	215	92	92	92
   query83	177	127	127	127
   query84	284	73	74	73
   query85	893	345	333	333
   query86	378	378	402	378
   query87	3598	3411	3387	3387
   query88	3247	2229	2253	2229
   query89	473	401	388	388
   query90	1941	216	207	207
   query91	165	128	127	127
   query92	59	55	56	55
   query93	1321	425	422	422
   query94	1300	183	186	183
   query95	526	498	463	463
   query96	615	318	317	317
   query97	4301	4194	4165	4165
   query98	208	198	188	188
   query99	1001	748	668	668
   Total cold run time: 294701 ms
   Total hot run time: 179537 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886964957

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886611651

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886964813

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886602499

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886656103

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.2 seconds
    stream load tsv:          573 seconds loaded 74807831229 Bytes, about 124 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          66 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          21.6 seconds inserted 10000000 Rows, about 462K ops/s
    storage size: 17184234905 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886632903

   TeamCity be ut coverage result:
    Function Coverage: 36.72% (8650/23558) 
    Line Coverage: 28.75% (70468/245121)
    Region Coverage: 27.67% (36414/131584)
    Branch Coverage: 24.39% (18639/76418)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/f729b362cfb648a1d73f79631b35751b264abb34_f729b362cfb648a1d73f79631b35751b264abb34/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886648066

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38821 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17968	5069	4991	4991
   q2	2047	142	141	141
   q3	10678	1123	1192	1123
   q4	10240	797	848	797
   q5	7783	3131	3141	3131
   q6	205	129	128	128
   q7	873	517	495	495
   q8	9238	1932	2001	1932
   q9	7353	6518	6474	6474
   q10	8247	3097	3067	3067
   q11	420	202	218	202
   q12	367	193	201	193
   q13	18123	3463	3440	3440
   q14	237	221	211	211
   q15	559	517	513	513
   q16	430	376	375	375
   q17	940	559	527	527
   q18	7332	6737	6687	6687
   q19	1592	1389	1404	1389
   q20	586	319	306	306
   q21	2788	2401	2410	2401
   q22	347	315	298	298
   Total cold run time: 108353 ms
   Total hot run time: 38821 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5007	5056	5050	5050
   q2	312	195	195	195
   q3	3343	3316	3321	3316
   q4	2256	2280	2260	2260
   q5	5825	5793	5810	5793
   q6	199	120	123	120
   q7	2314	1884	1863	1863
   q8	3448	3568	3586	3568
   q9	8900	8855	8775	8775
   q10	3761	3836	3836	3836
   q11	544	436	427	427
   q12	787	607	616	607
   q13	6448	3225	3200	3200
   q14	285	262	260	260
   q15	579	510	506	506
   q16	501	468	470	468
   q17	2051	2005	2020	2005
   q18	8778	8303	8277	8277
   q19	1626	1634	1633	1633
   q20	2167	1932	1937	1932
   q21	6140	5688	5723	5688
   q22	549	469	458	458
   Total cold run time: 65820 ms
   Total hot run time: 60237 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886682231

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit f729b362cfb648a1d73f79631b35751b264abb34 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
   Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886676269

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.28 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
   
   query1	0.06	0.06	0.05
   query2	0.06	0.02	0.02
   query3	0.25	0.12	0.10
   query4	1.78	0.12	0.12
   query5	0.52	0.52	0.52
   query6	1.38	0.64	0.65
   query7	0.02	0.01	0.01
   query8	0.03	0.03	0.02
   query9	0.56	0.51	0.49
   query10	0.55	0.55	0.56
   query11	0.12	0.09	0.09
   query12	0.11	0.09	0.09
   query13	0.61	0.59	0.60
   query14	0.78	0.80	0.77
   query15	0.83	0.80	0.79
   query16	0.34	0.35	0.35
   query17	0.97	1.00	1.02
   query18	0.26	0.26	0.26
   query19	1.86	1.79	1.78
   query20	0.01	0.01	0.01
   query21	15.43	0.58	0.60
   query22	2.81	2.10	1.60
   query23	17.16	0.77	0.72
   query24	16.05	0.64	0.59
   query25	2.16	0.15	0.15
   query26	0.14	0.13	0.13
   query27	0.14	0.15	0.14
   query28	6.33	0.84	0.79
   query29	12.55	3.20	3.17
   query30	0.52	0.50	0.49
   query31	2.78	0.34	0.36
   query32	3.38	0.48	0.49
   query33	3.24	3.21	3.20
   query34	15.92	4.19	4.18
   query35	4.18	4.17	4.17
   query36	1.11	1.05	1.05
   query37	0.06	0.04	0.04
   query38	0.04	0.03	0.03
   query39	0.03	0.01	0.01
   query40	0.17	0.13	0.14
   query41	0.08	0.02	0.01
   query42	0.02	0.02	0.01
   query43	0.03	0.02	0.02
   Total cold run time: 115.43 s
   Total hot run time: 30.28 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #29860:
URL: https://github.com/apache/doris/pull/29860


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886601334

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886612407

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org