You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/01/09 04:51:47 UTC

[PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

yiguolei opened a new pull request, #29704:
URL: https://github.com/apache/doris/pull/29704

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882458518

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882504581

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.41 seconds
    stream load tsv:          567 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          66 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          27.4 seconds inserted 10000000 Rows, about 364K ops/s
    storage size: 17183585684 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882415572

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29704:
URL: https://github.com/apache/doris/pull/29704


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882541082

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882492051

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 39562 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 680e424f260d6262b7b3fc83d5a5c1a1b702f3c8, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	18328	5560	5520	5520
   q2	2059	166	156	156
   q3	10616	1116	1193	1116
   q4	10206	791	799	791
   q5	7827	2995	2934	2934
   q6	206	134	136	134
   q7	905	532	500	500
   q8	9283	2010	1984	1984
   q9	6521	6400	6397	6397
   q10	8238	3067	3093	3067
   q11	435	235	223	223
   q12	392	233	241	233
   q13	18284	3494	3469	3469
   q14	236	213	212	212
   q15	554	517	505	505
   q16	476	409	393	393
   q17	937	748	673	673
   q18	7300	6728	6697	6697
   q19	1547	1519	1506	1506
   q20	741	316	314	314
   q21	2771	2444	2403	2403
   q22	387	338	335	335
   Total cold run time: 108249 ms
   Total hot run time: 39562 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5566	5496	5501	5496
   q2	359	270	255	255
   q3	3343	3285	3345	3285
   q4	2118	2136	2102	2102
   q5	5848	5836	5841	5836
   q6	210	122	126	122
   q7	2327	1902	1855	1855
   q8	3395	3551	3525	3525
   q9	8956	8985	8944	8944
   q10	3844	3896	3852	3852
   q11	597	522	514	514
   q12	819	649	648	648
   q13	6840	3278	3241	3241
   q14	302	258	253	253
   q15	544	529	514	514
   q16	558	488	503	488
   q17	2397	2398	2390	2390
   q18	8753	8722	8622	8622
   q19	1578	1605	1602	1602
   q20	2215	1967	1962	1962
   q21	5672	5483	5290	5290
   q22	566	519	508	508
   Total cold run time: 66807 ms
   Total hot run time: 61304 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882484483

   TeamCity be ut coverage result:
    Function Coverage: 36.66% (8639/23567) 
    Line Coverage: 28.68% (70232/244895)
    Region Coverage: 27.63% (36324/131470)
    Branch Coverage: 24.32% (18562/76330)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/680e424f260d6262b7b3fc83d5a5c1a1b702f3c8_680e424f260d6262b7b3fc83d5a5c1a1b702f3c8/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882502240

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 187040 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 680e424f260d6262b7b3fc83d5a5c1a1b702f3c8, data reload: false
   
   run tpcds-sf100 query with default conf and session variables
   query1	970	346	340	340
   query2	6869	2154	1997	1997
   query3	6726	230	228	228
   query4	25893	22624	22531	22531
   query5	6112	615	618	615
   query6	337	207	221	207
   query7	4635	306	318	306
   query8	269	240	240	240
   query9	9652	3349	3318	3318
   query10	595	349	345	345
   query11	16346	15740	15648	15648
   query12	183	118	120	118
   query13	1674	341	360	341
   query14	12738	7595	7584	7584
   query15	257	240	262	240
   query16	6356	363	323	323
   query17	1712	550	538	538
   query18	1980	322	335	322
   query19	240	172	171	171
   query20	135	121	130	121
   query21	189	95	103	95
   query22	4925	4584	4572	4572
   query23	32150	31459	31483	31459
   query24	12780	3057	3003	3003
   query25	616	399	391	391
   query26	1853	187	186	186
   query27	2919	315	318	315
   query28	7017	2175	2118	2118
   query29	2112	452	434	434
   query30	288	146	148	146
   query31	1070	873	855	855
   query32	141	102	104	102
   query33	847	360	364	360
   query34	872	495	499	495
   query35	1040	965	886	886
   query36	1382	1331	1323	1323
   query37	228	116	108	108
   query38	3464	3392	3333	3333
   query39	1397	1294	1302	1294
   query40	316	100	96	96
   query41	41	40	38	38
   query42	121	125	119	119
   query43	618	611	603	603
   query44	1307	833	843	833
   query45	220	215	209	209
   query46	1076	737	708	708
   query47	1755	1768	1752	1752
   query48	394	288	298	288
   query49	1216	347	334	334
   query50	732	350	375	350
   query51	5573	5481	5343	5343
   query52	132	108	111	108
   query53	248	194	191	191
   query54	1380	749	733	733
   query55	122	112	103	103
   query56	284	280	268	268
   query57	1057	1062	1054	1054
   query58	354	330	356	330
   query59	3272	3069	3132	3069
   query60	349	330	327	327
   query61	131	157	148	148
   query62	617	495	478	478
   query63	220	190	207	190
   query64	6120	1776	1755	1755
   query65	3461	3389	3422	3389
   query66	1336	468	453	453
   query67	15710	15769	15174	15174
   query68	12568	543	562	543
   query69	636	355	362	355
   query70	1908	1756	1751	1751
   query71	634	306	308	306
   query72	5446	3482	3504	3482
   query73	3035	348	344	344
   query74	7188	6643	6724	6643
   query75	5304	2314	2261	2261
   query76	6464	1130	1224	1130
   query77	754	357	335	335
   query78	9060	8869	8549	8549
   query79	1395	553	559	553
   query80	560	355	360	355
   query81	460	218	218	218
   query82	242	128	146	128
   query83	194	151	141	141
   query84	265	62	58	58
   query85	937	290	274	274
   query86	478	449	454	449
   query87	3657	3530	3468	3468
   query88	3565	2535	2539	2535
   query89	405	311	318	311
   query90	1923	277	253	253
   query91	121	95	96	95
   query92	115	91	99	91
   query93	1327	483	487	483
   query94	903	242	256	242
   query95	580	531	509	509
   query96	645	349	358	349
   query97	4417	4251	4351	4251
   query98	243	234	222	222
   query99	1156	905	905	905
   Total cold run time: 303716 ms
   Total hot run time: 187040 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882541048

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org