You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xingyingone (via GitHub)" <gi...@apache.org> on 2024/01/09 02:42:53 UTC

[PR] [improvement](executor) improve readability and efficiency of the checks [doris]

xingyingone opened a new pull request, #29696:
URL: https://github.com/apache/doris/pull/29696

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882337150

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882259878

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "adonis0147 (via GitHub)" <gi...@apache.org>.
adonis0147 commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445578359


##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
 
 bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
     std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
-    if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
-        query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
-    } else {
-        return false;
-    }

Review Comment:
   The original one can support failing fast and it can be more efficient than the modifications under some scenarios.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882336228

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.13 seconds
    stream load tsv:          569 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          66 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          27.3 seconds inserted 10000000 Rows, about 366K ops/s
    storage size: 17184131717 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29696:
URL: https://github.com/apache/doris/pull/29696


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882245561

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "adonis0147 (via GitHub)" <gi...@apache.org>.
adonis0147 commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445578359


##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
 
 bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
     std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
-    if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
-        query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
-    } else {
-        return false;
-    }

Review Comment:
   The original one can support failing fast and it can be more efficient than the modifications under some scenarios.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882301954

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38430 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17780	4943	4904	4904
   q2	2015	156	141	141
   q3	10586	1118	1200	1118
   q4	10457	819	830	819
   q5	7798	2967	2917	2917
   q6	202	122	125	122
   q7	899	514	496	496
   q8	9301	1968	1963	1963
   q9	6429	6361	6394	6361
   q10	8224	3050	2998	2998
   q11	429	209	210	209
   q12	380	229	231	229
   q13	18003	3374	3381	3374
   q14	239	213	207	207
   q15	544	504	508	504
   q16	434	393	396	393
   q17	948	706	588	588
   q18	7259	6632	6616	6616
   q19	1545	1507	1509	1507
   q20	704	297	291	291
   q21	2707	2415	2364	2364
   q22	359	309	323	309
   Total cold run time: 107242 ms
   Total hot run time: 38430 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4891	4930	4912	4912
   q2	331	250	243	243
   q3	3265	3283	3246	3246
   q4	2096	2002	1996	1996
   q5	5769	5746	5742	5742
   q6	204	113	117	113
   q7	2304	1884	1869	1869
   q8	3360	3425	3452	3425
   q9	8787	8716	8743	8716
   q10	3779	3835	3842	3835
   q11	544	447	458	447
   q12	798	626	628	626
   q13	6147	3176	3178	3176
   q14	295	278	255	255
   q15	544	504	494	494
   q16	554	496	494	494
   q17	1841	1848	1856	1848
   q18	8553	8369	8217	8217
   q19	1579	1637	1611	1611
   q20	2176	1942	1934	1934
   q21	5622	5187	5245	5187
   q22	502	460	477	460
   Total cold run time: 63941 ms
   Total hot run time: 58846 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445579831


##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
 
 bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
     std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
-    if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
-        query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
-    } else {
-        return false;
-    }

Review Comment:
   > The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
   The process for failure scenarios is the same
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882284064

   TeamCity be ut coverage result:
    Function Coverage: 36.66% (8639/23566) 
    Line Coverage: 28.68% (70226/244842)
    Region Coverage: 27.63% (36322/131448)
    Branch Coverage: 24.31% (18557/76320)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/89887da04e9321a61548abc5c67064507a00d434_89887da04e9321a61548abc5c67064507a00d434/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882337086

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882325079

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 184763 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false
   
   run tpcds-sf100 query with default conf and session variables
   query1	937	339	338	338
   query2	6567	2011	1923	1923
   query3	6716	234	225	225
   query4	26830	22558	22555	22555
   query5	5769	603	583	583
   query6	334	230	211	211
   query7	4655	307	288	288
   query8	273	241	231	231
   query9	8743	2947	2948	2947
   query10	554	336	313	313
   query11	16320	15789	15757	15757
   query12	170	114	98	98
   query13	1693	351	343	343
   query14	13543	7757	7529	7529
   query15	268	245	216	216
   query16	6437	340	334	334
   query17	1701	522	535	522
   query18	1950	309	300	300
   query19	248	165	165	165
   query20	105	97	105	97
   query21	188	95	95	95
   query22	4908	5012	4917	4917
   query23	32567	31601	31516	31516
   query24	12237	2994	2992	2992
   query25	600	368	360	360
   query26	1843	183	171	171
   query27	3027	312	302	302
   query28	6942	2135	2123	2123
   query29	2063	421	419	419
   query30	297	145	150	145
   query31	1021	827	820	820
   query32	130	86	88	86
   query33	840	344	343	343
   query34	846	480	471	471
   query35	1001	918	866	866
   query36	1377	1245	1295	1245
   query37	239	93	88	88
   query38	3441	3362	3359	3359
   query39	1341	1319	1459	1319
   query40	302	92	89	89
   query41	38	35	36	35
   query42	114	111	110	110
   query43	583	532	530	530
   query44	1207	808	815	808
   query45	215	206	212	206
   query46	1063	722	743	722
   query47	1767	1683	1610	1610
   query48	367	320	284	284
   query49	1207	322	322	322
   query50	740	339	328	328
   query51	5452	5270	5309	5270
   query52	123	97	99	97
   query53	242	175	170	170
   query54	1419	687	699	687
   query55	116	106	102	102
   query56	304	245	288	245
   query57	1033	1024	972	972
   query58	328	294	290	290
   query59	3087	2777	2793	2777
   query60	348	302	305	302
   query61	133	146	149	146
   query62	571	500	484	484
   query63	199	183	188	183
   query64	6033	1710	1718	1710
   query65	3428	3358	3364	3358
   query66	1372	362	415	362
   query67	15699	15569	15349	15349
   query68	13012	541	547	541
   query69	597	302	325	302
   query70	1796	1605	1654	1605
   query71	606	296	258	258
   query72	5304	3400	3412	3400
   query73	3045	339	344	339
   query74	6963	6439	6560	6439
   query75	5020	2328	2292	2292
   query76	6303	1059	1178	1059
   query77	749	303	311	303
   query78	9020	8603	8507	8507
   query79	1093	535	531	531
   query80	574	347	347	347
   query81	462	213	213	213
   query82	241	112	106	106
   query83	188	143	140	140
   query84	250	57	57	57
   query85	954	281	259	259
   query86	436	445	413	413
   query87	3563	3449	3430	3430
   query88	3085	2513	2532	2513
   query89	375	283	269	269
   query90	1890	267	253	253
   query91	122	95	97	95
   query92	81	89	69	69
   query93	1139	503	503	503
   query94	913	249	224	224
   query95	586	510	468	468
   query96	637	341	335	335
   query97	4336	4231	4338	4231
   query98	238	224	220	220
   query99	1171	898	882	882
   Total cold run time: 300960 ms
   Total hot run time: 184763 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]

Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445579831


##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
 
 bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
     std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
-    if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
-        query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
-    } else {
-        return false;
-    }

Review Comment:
   > The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
   The process for failure scenarios is the same
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org