You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/03/04 13:31:03 UTC

[PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

liaoxin01 opened a new pull request, #31758:
URL: https://github.com/apache/doris/pull/31758

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   cherry-pick from: #31532 
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976692643

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c with default session variables
   Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
   Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       20.7 seconds inserted 10000000 Rows, about 483K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637388

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976645884

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 49892 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17725	4348	4367	4348
   q2	2053	155	138	138
   q3	10505	1927	1974	1927
   q4	10332	1233	1334	1233
   q5	8476	3944	3977	3944
   q6	232	122	126	122
   q7	2048	1604	1572	1572
   q8	9254	2709	2723	2709
   q9	10958	10539	10601	10539
   q10	8639	3544	3515	3515
   q11	423	235	247	235
   q12	470	301	303	301
   q13	18367	3935	4037	3935
   q14	344	322	330	322
   q15	498	453	454	453
   q16	687	590	600	590
   q17	1108	948	969	948
   q18	7334	6759	6872	6759
   q19	1680	1541	1511	1511
   q20	494	298	305	298
   q21	4477	4129	4095	4095
   q22	490	410	398	398
   Total cold run time: 116594 ms
   Total hot run time: 49892 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4301	4288	4329	4288
   q2	310	218	222	218
   q3	4161	4119	4150	4119
   q4	2721	2723	2752	2723
   q5	7200	7146	7217	7146
   q6	235	118	119	118
   q7	3221	2777	2837	2777
   q8	4333	4388	4436	4388
   q9	17507	17185	17222	17185
   q10	4264	4272	4295	4272
   q11	736	664	680	664
   q12	1015	847	848	847
   q13	7043	3730	3731	3730
   q14	465	419	430	419
   q15	504	452	447	447
   q16	748	695	699	695
   q17	3843	3813	3890	3813
   q18	8787	8689	8787	8689
   q19	1717	1725	1691	1691
   q20	2442	2125	2124	2124
   q21	8529	8518	8519	8518
   q22	1079	996	997	996
   Total cold run time: 85161 ms
   Total hot run time: 79867 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976686025

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 29.7 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
   
   query1	0.02	0.02	0.02
   query2	0.06	0.03	0.02
   query3	0.24	0.05	0.04
   query4	1.84	0.06	0.07
   query5	0.53	0.52	0.52
   query6	1.24	0.60	0.60
   query7	0.01	0.01	0.01
   query8	0.03	0.02	0.02
   query9	0.52	0.49	0.46
   query10	0.54	0.54	0.54
   query11	0.11	0.08	0.08
   query12	0.11	0.08	0.09
   query13	0.62	0.62	0.61
   query14	0.79	0.79	0.79
   query15	0.80	0.75	0.76
   query16	0.37	0.40	0.36
   query17	1.01	1.02	0.98
   query18	0.22	0.27	0.22
   query19	1.92	1.78	1.83
   query20	0.02	0.01	0.00
   query21	15.47	0.55	0.53
   query22	2.00	2.56	0.81
   query23	17.31	1.06	0.92
   query24	4.85	1.63	1.12
   query25	0.38	0.12	0.04
   query26	0.59	0.13	0.14
   query27	0.04	0.04	0.03
   query28	7.86	0.76	0.69
   query29	12.62	2.26	2.12
   query30	0.58	0.58	0.52
   query31	2.81	0.40	0.38
   query32	3.41	0.52	0.49
   query33	3.07	3.05	3.05
   query34	15.25	4.85	4.83
   query35	4.85	4.86	4.83
   query36	1.05	1.01	1.02
   query37	0.05	0.05	0.05
   query38	0.04	0.02	0.02
   query39	0.01	0.01	0.01
   query40	0.16	0.14	0.14
   query41	0.07	0.01	0.02
   query42	0.02	0.01	0.01
   query43	0.02	0.02	0.02
   Total cold run time: 103.51 s
   Total hot run time: 29.7 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976588433

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976674909

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 201665 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
   
   query1	920	389	377	377
   query2	6523	2201	2126	2126
   query3	7378	253	219	219
   query4	23202	18307	17821	17821
   query5	19721	6495	6497	6495
   query6	411	216	224	216
   query7	5248	300	297	297
   query8	246	249	229	229
   query9	3083	2659	2588	2588
   query10	477	278	290	278
   query11	11385	10589	10665	10589
   query12	119	76	64	64
   query13	5623	628	604	604
   query14	19132	13720	13729	13720
   query15	358	235	231	231
   query16	6418	266	257	257
   query17	1334	1522	842	842
   query18	2307	427	404	404
   query19	224	139	147	139
   query20	68	73	74	73
   query21	183	99	87	87
   query22	5138	5027	5043	5027
   query23	32446	31988	31824	31824
   query24	6747	6450	6481	6450
   query25	515	425	409	409
   query26	518	163	155	155
   query27	1684	289	294	289
   query28	6009	2204	2177	2177
   query29	2810	2698	2685	2685
   query30	236	164	157	157
   query31	882	710	739	710
   query32	64	58	54	54
   query33	385	250	248	248
   query34	854	466	484	466
   query35	1103	955	916	916
   query36	1542	1769	1563	1563
   query37	86	60	58	58
   query38	3085	2926	2982	2926
   query39	1365	1319	1315	1315
   query40	195	93	96	93
   query41	35	32	32	32
   query42	82	85	84	84
   query43	570	547	559	547
   query44	1115	706	710	706
   query45	239	224	227	224
   query46	1221	943	987	943
   query47	1782	1818	1866	1818
   query48	999	669	660	660
   query49	609	378	359	359
   query50	845	611	608	608
   query51	4779	4693	4622	4622
   query52	91	79	67	67
   query53	444	323	313	313
   query54	2629	2448	2482	2448
   query55	87	72	78	72
   query56	212	196	200	196
   query57	1114	1132	1241	1132
   query58	213	180	190	180
   query59	3621	3166	3206	3166
   query60	231	214	203	203
   query61	84	82	85	82
   query62	816	577	485	485
   query63	472	345	345	345
   query64	2505	1518	1282	1282
   query65	3656	3566	3564	3564
   query66	758	361	363	361
   query67	18038	15347	15950	15347
   query68	8171	628	645	628
   query69	583	354	356	354
   query70	1895	1835	1536	1536
   query71	403	302	305	302
   query72	6430	3421	3465	3421
   query73	715	322	310	310
   query74	6419	5842	5848	5842
   query75	4577	3710	3630	3630
   query76	4638	1172	1221	1172
   query77	562	240	252	240
   query78	13360	12439	11477	11477
   query79	10340	643	640	640
   query80	910	397	388	388
   query81	502	229	231	229
   query82	1136	101	92	92
   query83	158	132	130	130
   query84	260	70	66	66
   query85	753	295	284	284
   query86	392	371	419	371
   query87	3250	2993	3021	2993
   query88	4951	2329	2318	2318
   query89	442	312	294	294
   query90	1854	199	198	198
   query91	146	114	114	114
   query92	53	49	49	49
   query93	5911	561	561	561
   query94	744	197	195	195
   query95	1092	1048	1052	1048
   query96	647	324	333	324
   query97	6471	6368	6431	6368
   query98	187	179	181	179
   query99	2928	928	888	888
   Total cold run time: 318151 ms
   Total hot run time: 201665 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #31758:
URL: https://github.com/apache/doris/pull/31758


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637368

   TeamCity be ut coverage result:
    Function Coverage: 37.82% (8040/21256) 
    Line Coverage: 29.50% (65572/222281)
    Region Coverage: 28.96% (33736/116482)
    Branch Coverage: 24.82% (17319/69776)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/31c2eabf77ccf5e7c7229ae6a76a249ba35e368c_31c2eabf77ccf5e7c7229ae6a76a249ba35e368c/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637273

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976588551

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976600915

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org