You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "nextdreamblue (via GitHub)" <gi...@apache.org> on 2024/01/03 11:05:35 UTC

[PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

nextdreamblue opened a new pull request, #29479:
URL: https://github.com/apache/doris/pull/29479

   
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29479:
URL: https://github.com/apache/doris/pull/29479


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875211962

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875242362

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
   ```
   Tpch sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	5491	5128	5157	5128
   q2	390	156	158	156
   q3	1459	1186	1199	1186
   q4	1096	870	793	793
   q5	3115	3086	3116	3086
   q6	225	137	132	132
   q7	996	551	518	518
   q8	2158	2216	2207	2207
   q9	6708	6634	6669	6634
   q10	3183	3176	3094	3094
   q11	325	218	214	214
   q12	384	229	243	229
   q13	4397	3600	3603	3600
   q14	243	225	212	212
   q15	600	556	557	556
   q16	452	410	391	391
   q17	1049	542	487	487
   q18	7109	6706	6769	6706
   q19	1659	1491	1605	1491
   q20	578	342	355	342
   q21	2838	2499	2445	2445
   q22	399	339	317	317
   Total cold run time: 44854 ms
   Total hot run time: 39924 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	5199	5119	5094	5094
   q2	336	240	261	240
   q3	3351	3331	3353	3331
   q4	2145	2003	1996	1996
   q5	5943	5920	5950	5920
   q6	232	130	126	126
   q7	2358	1909	1868	1868
   q8	3582	3650	3630	3630
   q9	9052	9015	8980	8980
   q10	3878	3936	3939	3936
   q11	567	480	481	480
   q12	792	646	651	646
   q13	3918	3191	3185	3185
   q14	313	279	295	279
   q15	615	545	528	528
   q16	589	531	528	528
   q17	2008	1831	1810	1810
   q18	8800	8345	8376	8345
   q19	1754	1651	1680	1651
   q20	2285	2002	1981	1981
   q21	5673	5409	5337	5337
   q22	582	522	490	490
   Total cold run time: 63972 ms
   Total hot run time: 60381 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875216832

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
   ```
   Tpch sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	18761	5445	5046	5046
   q2	2035	157	148	148
   q3	10605	1159	1093	1093
   q4	10204	755	900	755
   q5	7785	2946	2977	2946
   q6	209	137	131	131
   q7	949	517	534	517
   q8	9345	2018	2007	2007
   q9	6838	6411	6400	6400
   q10	8246	3015	3042	3015
   q11	430	222	221	221
   q12	389	236	231	231
   q13	18000	3614	3601	3601
   q14	244	222	213	213
   q15	574	551	534	534
   q16	459	399	401	399
   q17	953	526	579	526
   q18	7404	6847	6774	6774
   q19	1583	1297	1313	1297
   q20	744	334	356	334
   q21	2863	2345	2346	2345
   q22	368	306	331	306
   Total cold run time: 108988 ms
   Total hot run time: 38839 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5143	5102	5109	5102
   q2	338	236	242	236
   q3	3324	3292	3265	3265
   q4	2162	2045	2022	2022
   q5	5826	5815	5779	5779
   q6	211	124	121	121
   q7	2311	1945	1903	1903
   q8	3377	3417	3456	3417
   q9	8843	8755	8773	8755
   q10	3798	3821	3872	3821
   q11	577	481	495	481
   q12	823	635	616	616
   q13	10699	3196	3190	3190
   q14	280	272	267	267
   q15	588	531	535	531
   q16	580	480	517	480
   q17	1941	1777	1762	1762
   q18	8759	8449	8255	8255
   q19	1617	1621	1618	1618
   q20	2254	1987	1970	1970
   q21	5502	5286	5242	5242
   q22	587	521	492	492
   Total cold run time: 69540 ms
   Total hot run time: 59325 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "nextdreamblue (via GitHub)" <gi...@apache.org>.
nextdreamblue commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875192616

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875212000

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875230387

   
   TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
   ```
   TPC-DS sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
   
   run tpcds-sf100 query with default conf and session variables
   query1	889	345	336	336
   query2	5122	1907	1873	1873
   query3	5185	205	197	197
   query4	24009	22452	22288	22288
   query5	3235	508	505	505
   query6	254	193	188	188
   query7	4091	289	294	289
   query8	251	223	216	216
   query9	7365	2742	2657	2657
   query10	337	237	237	237
   query11	16026	15362	15731	15362
   query12	129	76	73	73
   query13	1507	329	319	319
   query14	10512	7411	7340	7340
   query15	309	218	225	218
   query16	5673	269	271	269
   query17	1613	523	530	523
   query18	1782	289	263	263
   query19	180	155	153	153
   query20	83	76	82	76
   query21	159	101	101	101
   query22	5081	4820	4507	4507
   query23	32243	31643	31202	31202
   query24	11710	3457	3423	3423
   query25	624	374	374	374
   query26	1603	158	157	157
   query27	2709	288	291	288
   query28	6277	1966	1942	1942
   query29	1810	422	407	407
   query30	305	149	156	149
   query31	1118	777	820	777
   query32	86	59	54	54
   query33	673	292	270	270
   query34	841	444	435	435
   query35	968	783	809	783
   query36	1350	1282	1167	1167
   query37	175	74	75	74
   query38	3456	3306	3274	3274
   query39	1341	1307	1285	1285
   query40	286	92	93	92
   query41	38	36	35	35
   query42	102	89	93	89
   query43	519	500	479	479
   query44	1035	706	717	706
   query45	232	224	223	223
   query46	1167	781	748	748
   query47	1652	1574	1581	1574
   query48	370	277	281	277
   query49	1139	331	329	329
   query50	1077	877	831	831
   query51	5361	5261	5418	5261
   query52	91	91	90	90
   query53	217	185	179	179
   query54	1581	607	599	599
   query55	95	86	85	85
   query56	209	204	192	192
   query57	1049	965	996	965
   query58	228	216	207	207
   query59	2703	2617	2480	2480
   query60	244	238	242	238
   query61	84	86	87	86
   query62	1457	509	507	507
   query63	188	173	165	165
   query64	5775	1755	1790	1755
   query65	3376	3283	3263	3263
   query66	1154	344	324	324
   query67	15898	15189	15232	15189
   query68	11800	539	561	539
   query69	498	256	243	243
   query70	1695	1531	1546	1531
   query71	485	226	224	224
   query72	6468	4094	4067	4067
   query73	2778	311	322	311
   query74	6857	6489	6505	6489
   query75	5263	2257	2312	2257
   query76	6091	1190	1127	1127
   query77	667	293	310	293
   query78	9127	8864	8553	8553
   query79	5500	536	535	535
   query80	747	391	393	391
   query81	483	215	215	215
   query82	196	103	103	103
   query83	185	142	140	140
   query84	244	53	52	52
   query85	879	281	283	281
   query86	397	412	429	412
   query87	3546	3522	3394	3394
   query88	3713	2292	2289	2289
   query89	358	295	293	293
   query90	1798	216	210	210
   query91	126	93	94	93
   query92	59	60	51	51
   query93	3396	442	489	442
   query94	761	189	188	188
   query95	472	405	394	394
   query96	636	321	319	319
   query97	4262	4179	4150	4150
   query98	215	202	190	190
   query99	1579	936	889	889
   Total cold run time: 289689 ms
   Total hot run time: 181684 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875240752

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 47.11 seconds
    stream load tsv:          561 seconds loaded 74807831229 Bytes, about 127 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          66 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          27.2 seconds inserted 10000000 Rows, about 367K ops/s
    storage size: 17183276427 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org