You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/10/09 09:45:04 UTC

[PR] [improve](binlog) Filter the truncated partitions [doris]

w41ter opened a new pull request, #41611:
URL: https://github.com/apache/doris/pull/41611

   The suite case in https://github.com/selectdb/ccr-syncer/pull/190
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401951620

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401847515

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401950020

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 191398 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 552a9dab74c2cc94bfe67999bd336e4f46a1f8a9, data reload: false
   
   query1	959	400	407	400
   query2	6252	2120	2092	2092
   query3	8689	196	204	196
   query4	34201	23515	23494	23494
   query5	3614	491	469	469
   query6	271	163	165	163
   query7	4199	302	325	302
   query8	279	210	204	204
   query9	9425	2671	2656	2656
   query10	445	274	271	271
   query11	17686	15211	15303	15211
   query12	132	101	96	96
   query13	1573	486	453	453
   query14	9356	6464	7679	6464
   query15	253	170	182	170
   query16	7531	451	435	435
   query17	1688	608	601	601
   query18	1822	335	318	318
   query19	371	162	163	162
   query20	125	116	112	112
   query21	216	107	105	105
   query22	4784	4686	4710	4686
   query23	35001	34237	34092	34092
   query24	11029	2872	2872	2872
   query25	679	432	409	409
   query26	1210	163	168	163
   query27	2384	309	291	291
   query28	7255	2430	2421	2421
   query29	801	437	422	422
   query30	258	155	158	155
   query31	1040	767	830	767
   query32	98	55	54	54
   query33	774	293	304	293
   query34	900	496	506	496
   query35	855	744	745	744
   query36	1108	949	917	917
   query37	158	87	86	86
   query38	4061	3883	3961	3883
   query39	1466	1451	1396	1396
   query40	214	97	98	97
   query41	47	44	43	43
   query42	113	97	93	93
   query43	517	492	496	492
   query44	1208	802	824	802
   query45	199	165	167	165
   query46	1150	726	726	726
   query47	1967	1815	1879	1815
   query48	449	363	348	348
   query49	892	396	404	396
   query50	818	422	420	420
   query51	7153	6891	6965	6891
   query52	97	86	87	86
   query53	252	178	179	178
   query54	1292	468	477	468
   query55	81	79	79	79
   query56	288	272	283	272
   query57	1222	1143	1186	1143
   query58	239	229	235	229
   query59	3194	3084	2969	2969
   query60	282	269	291	269
   query61	112	102	99	99
   query62	864	676	677	676
   query63	215	187	181	181
   query64	4244	667	635	635
   query65	3229	3218	3205	3205
   query66	1079	303	317	303
   query67	15826	15660	15604	15604
   query68	4415	567	563	563
   query69	551	295	289	289
   query70	1175	1073	1098	1073
   query71	385	271	267	267
   query72	7638	4059	3986	3986
   query73	765	342	352	342
   query74	10195	8865	8951	8865
   query75	3761	2674	2669	2669
   query76	3382	922	986	922
   query77	749	302	300	300
   query78	10407	9553	9587	9553
   query79	1867	605	606	605
   query80	991	440	460	440
   query81	569	240	241	240
   query82	644	138	138	138
   query83	368	156	134	134
   query84	281	77	78	77
   query85	1388	298	283	283
   query86	425	280	295	280
   query87	4463	4299	4233	4233
   query88	3119	2436	2393	2393
   query89	408	288	281	281
   query90	2020	183	182	182
   query91	154	109	107	107
   query92	60	47	47	47
   query93	2183	537	538	537
   query94	1088	288	296	288
   query95	360	251	250	250
   query96	616	276	273	273
   query97	3230	3088	3111	3088
   query98	212	194	196	194
   query99	1551	1294	1314	1294
   Total cold run time: 299611 ms
   Total hot run time: 191398 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401951676

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #41611:
URL: https://github.com/apache/doris/pull/41611


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401927028

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 40690 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 552a9dab74c2cc94bfe67999bd336e4f46a1f8a9, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17572	7411	7260	7260
   q2	2038	280	266	266
   q3	12145	1091	1194	1091
   q4	10556	707	750	707
   q5	7726	2800	2808	2800
   q6	234	152	149	149
   q7	991	624	631	624
   q8	9363	1932	2012	1932
   q9	6567	6420	6376	6376
   q10	6969	2286	2310	2286
   q11	435	246	242	242
   q12	407	216	221	216
   q13	17791	2984	3014	2984
   q14	261	208	215	208
   q15	562	526	511	511
   q16	638	582	586	582
   q17	957	563	541	541
   q18	7127	6681	6740	6681
   q19	1356	995	895	895
   q20	465	202	206	202
   q21	4006	3263	3170	3170
   q22	1114	967	1004	967
   Total cold run time: 109280 ms
   Total hot run time: 40690 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	7243	7226	7185	7185
   q2	329	226	230	226
   q3	2969	2897	2972	2897
   q4	2035	1837	1803	1803
   q5	5789	5751	5750	5750
   q6	229	150	143	143
   q7	2222	1860	1856	1856
   q8	3337	3540	3461	3461
   q9	8962	8852	8783	8783
   q10	3572	3537	3565	3537
   q11	590	486	499	486
   q12	834	663	632	632
   q13	10523	3125	3175	3125
   q14	311	277	273	273
   q15	570	520	523	520
   q16	686	640	659	640
   q17	1829	1627	1610	1610
   q18	8310	7908	7426	7426
   q19	1729	1510	1527	1510
   q20	2120	1861	1892	1861
   q21	5470	5338	5401	5338
   q22	1121	1037	1042	1037
   Total cold run time: 70780 ms
   Total hot run time: 60099 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401847302

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
   See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improve](binlog) Filter the truncated partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41611:
URL: https://github.com/apache/doris/pull/41611#issuecomment-2401960915

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 32.84 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 552a9dab74c2cc94bfe67999bd336e4f46a1f8a9, data reload: false
   
   query1	0.03	0.03	0.04
   query2	0.07	0.02	0.03
   query3	0.22	0.06	0.06
   query4	1.65	0.10	0.10
   query5	0.50	0.50	0.48
   query6	1.13	0.73	0.73
   query7	0.02	0.01	0.01
   query8	0.04	0.03	0.03
   query9	0.56	0.50	0.50
   query10	0.58	0.56	0.55
   query11	0.14	0.10	0.10
   query12	0.13	0.11	0.10
   query13	0.60	0.59	0.59
   query14	2.70	2.74	2.71
   query15	0.89	0.83	0.85
   query16	0.38	0.39	0.39
   query17	1.03	0.98	1.03
   query18	0.21	0.19	0.20
   query19	1.95	1.86	1.93
   query20	0.03	0.01	0.02
   query21	15.36	0.60	0.60
   query22	2.37	2.81	1.85
   query23	17.09	0.81	0.91
   query24	2.89	1.89	1.24
   query25	0.20	0.13	0.03
   query26	0.61	0.14	0.13
   query27	0.04	0.05	0.04
   query28	10.08	1.08	1.07
   query29	12.54	3.17	3.19
   query30	0.24	0.05	0.06
   query31	2.88	0.39	0.38
   query32	3.26	0.47	0.46
   query33	2.97	3.05	3.02
   query34	16.68	4.47	4.46
   query35	4.54	4.48	4.48
   query36	0.67	0.48	0.49
   query37	0.07	0.06	0.06
   query38	0.04	0.03	0.03
   query39	0.03	0.02	0.02
   query40	0.16	0.13	0.12
   query41	0.07	0.02	0.02
   query42	0.04	0.02	0.02
   query43	0.03	0.03	0.03
   Total cold run time: 105.72 s
   Total hot run time: 32.84 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org