You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/02/28 09:04:00 UTC

[PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

liaoxin01 opened a new pull request, #31532:
URL: https://github.com/apache/doris/pull/31532

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   [INTERNAL_ERROR][INTERNAL_ERROR]close wait failed coz rpc error. VNodeChannel[4310428608-10003], load_id=c478455147aeb7f-847eb9b9965435a7, txn_id=42764708, node=10.86.1.83:8060, add batch req success but status isn't ok, err: [INTERNAL_ERROR]PStatus: (10.86.1.83)[INTERNAL_ERROR]unknown tablet to append data, tablet=15461331
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968747262

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968527099

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968578306

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 169506 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
   
   query1	941	337	335	335
   query2	6521	1746	1658	1658
   query3	6720	210	210	210
   query4	23408	20590	20626	20590
   query5	4223	371	373	371
   query6	258	170	160	160
   query7	4618	301	303	301
   query8	274	194	196	194
   query9	8412	2260	2243	2243
   query10	417	211	218	211
   query11	14675	14164	14180	14164
   query12	132	83	82	82
   query13	1653	426	437	426
   query14	8277	6637	6532	6532
   query15	197	178	186	178
   query16	7089	262	248	248
   query17	1043	532	502	502
   query18	1894	271	265	265
   query19	193	146	144	144
   query20	87	81	86	81
   query21	198	132	117	117
   query22	4789	4668	4547	4547
   query23	30571	29821	29951	29821
   query24	11824	3131	3119	3119
   query25	654	356	345	345
   query26	1782	152	158	152
   query27	3018	317	324	317
   query28	7008	1812	1804	1804
   query29	1159	563	566	563
   query30	284	132	139	132
   query31	869	685	706	685
   query32	94	58	56	56
   query33	725	220	219	219
   query34	1046	482	494	482
   query35	836	732	748	732
   query36	926	881	849	849
   query37	134	61	61	61
   query38	3152	3007	2960	2960
   query39	1318	1290	1261	1261
   query40	280	101	104	101
   query41	38	36	37	36
   query42	103	98	101	98
   query43	453	425	427	425
   query44	1055	708	728	708
   query45	197	183	172	172
   query46	1036	786	792	786
   query47	1543	1453	1495	1453
   query48	434	346	343	343
   query49	1123	285	296	285
   query50	783	383	388	383
   query51	4377	4341	4260	4260
   query52	97	93	89	89
   query53	330	266	264	264
   query54	299	226	224	224
   query55	87	80	86	80
   query56	224	197	194	194
   query57	959	883	872	872
   query58	217	201	186	186
   query59	2260	2085	2101	2085
   query60	242	215	214	214
   query61	90	84	82	82
   query62	607	367	360	360
   query63	280	263	251	251
   query64	6578	3047	3109	3047
   query65	3221	3209	3224	3209
   query66	1443	327	310	310
   query67	14490	14371	14130	14130
   query68	5004	558	572	558
   query69	535	364	378	364
   query70	1276	1225	1206	1206
   query71	411	255	268	255
   query72	6230	2785	2656	2656
   query73	707	320	334	320
   query74	6304	5960	5959	5959
   query75	3075	2455	2410	2410
   query76	2789	1025	1141	1025
   query77	352	227	223	223
   query78	9068	8496	8492	8492
   query79	1125	531	523	523
   query80	813	357	341	341
   query81	465	194	193	193
   query82	1305	84	84	84
   query83	205	120	121	120
   query84	278	80	82	80
   query85	1189	340	330	330
   query86	337	297	303	297
   query87	3349	3109	3091	3091
   query88	2793	2332	2323	2323
   query89	388	335	324	324
   query90	2037	164	169	164
   query91	151	119	122	119
   query92	56	47	51	47
   query93	1031	521	510	510
   query94	1113	180	177	177
   query95	437	332	348	332
   query96	573	270	265	265
   query97	4248	4160	4151	4151
   query98	218	202	186	186
   query99	1043	617	691	617
   Total cold run time: 263170 ms
   Total hot run time: 169506 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968587860

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 32.1 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
   
   query1	0.04	0.03	0.03
   query2	0.05	0.02	0.02
   query3	0.23	0.06	0.06
   query4	1.68	0.10	0.10
   query5	0.52	0.52	0.52
   query6	1.25	0.66	0.69
   query7	0.01	0.01	0.02
   query8	0.03	0.02	0.02
   query9	0.55	0.53	0.53
   query10	0.56	0.56	0.58
   query11	0.13	0.10	0.09
   query12	0.12	0.10	0.10
   query13	0.63	0.61	0.61
   query14	0.79	0.79	0.82
   query15	0.84	0.82	0.80
   query16	0.37	0.37	0.39
   query17	0.99	1.00	0.96
   query18	0.27	0.25	0.25
   query19	1.77	1.74	1.73
   query20	0.01	0.01	0.02
   query21	15.41	0.62	0.62
   query22	2.58	3.35	2.45
   query23	17.64	1.06	1.04
   query24	2.16	0.24	0.12
   query25	0.23	0.12	0.05
   query26	0.16	0.15	0.15
   query27	0.05	0.04	0.04
   query28	12.81	0.83	0.83
   query29	12.60	3.24	3.31
   query30	0.57	0.54	0.54
   query31	2.79	0.35	0.35
   query32	3.36	0.48	0.48
   query33	3.16	3.22	3.21
   query34	15.36	4.73	4.73
   query35	4.74	4.71	4.71
   query36	1.13	1.07	1.07
   query37	0.06	0.04	0.04
   query38	0.04	0.03	0.02
   query39	0.02	0.01	0.01
   query40	0.18	0.13	0.13
   query41	0.07	0.02	0.02
   query42	0.02	0.02	0.01
   query43	0.03	0.02	0.02
   Total cold run time: 106.01 s
   Total hot run time: 32.1 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31532:
URL: https://github.com/apache/doris/pull/31532


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968559373

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38019 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	18133	4183	4120	4120
   q2	2603	146	140	140
   q3	11000	983	966	966
   q4	5041	971	998	971
   q5	7973	3000	3075	3000
   q6	185	126	122	122
   q7	1294	788	777	777
   q8	9638	1968	2072	1968
   q9	7223	6267	6263	6263
   q10	8178	2481	2489	2481
   q11	421	202	216	202
   q12	798	318	309	309
   q13	17973	3263	3207	3207
   q14	285	262	251	251
   q15	530	499	487	487
   q16	470	418	412	412
   q17	941	892	886	886
   q18	6577	6126	5811	5811
   q19	1569	1510	1511	1510
   q20	549	287	279	279
   q21	6781	3583	3577	3577
   q22	801	283	280	280
   Total cold run time: 108963 ms
   Total hot run time: 38019 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4008	3983	4007	3983
   q2	300	215	211	211
   q3	3001	2927	2949	2927
   q4	1868	1813	1805	1805
   q5	5084	5126	5125	5125
   q6	200	114	114	114
   q7	2130	1699	1717	1699
   q8	3105	3182	3197	3182
   q9	8258	8237	8267	8237
   q10	6213	3624	3628	3624
   q11	517	433	437	433
   q12	713	517	521	517
   q13	3826	3069	3061	3061
   q14	276	242	256	242
   q15	532	492	487	487
   q16	502	466	476	466
   q17	1700	1678	1677	1677
   q18	7839	7535	7391	7391
   q19	1630	1643	1621	1621
   q20	2094	1910	1907	1907
   q21	4757	4640	4593	4593
   q22	513	455	462	455
   Total cold run time: 59066 ms
   Total hot run time: 53757 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968543992

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968582939

   TeamCity be ut coverage result:
    Function Coverage: 35.69% (8548/23952) 
    Line Coverage: 27.49% (69386/252411)
    Region Coverage: 26.65% (35974/135007)
    Branch Coverage: 23.46% (18389/78394)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/c68475eebda09b3c32794775d1cc9036db9e76eb_c68475eebda09b3c32794775d1cc9036db9e76eb/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968527487

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968593620

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb with default session variables
   Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
   Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
   Insert into select:       16.8 seconds inserted 10000000 Rows, about 595K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968747199

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org