You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2024/03/08 17:02:08 UTC

[PR] [Function](lib) use libbase64 to do base64 [doris]

HappenLee opened a new pull request, #32019:
URL: https://github.com/apache/doris/pull/32019

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987103620

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 186146 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 66d444e9ec35e664ac6e34997d2a39452cd519f0, data reload: false
   
   query1	924	360	334	334
   query2	7353	2041	2028	2028
   query3	6713	212	211	211
   query4	25886	20832	21157	20832
   query5	4296	539	444	444
   query6	282	186	187	186
   query7	4636	327	309	309
   query8	242	181	171	171
   query9	8337	2254	2239	2239
   query10	449	237	252	237
   query11	14918	14352	14334	14334
   query12	135	93	86	86
   query13	1642	413	435	413
   query14	10440	10121	9626	9626
   query15	239	198	188	188
   query16	7811	285	283	283
   query17	1285	589	582	582
   query18	1975	316	289	289
   query19	205	167	163	163
   query20	92	87	90	87
   query21	204	127	127	127
   query22	4698	4410	4414	4410
   query23	31619	31088	30920	30920
   query24	12221	3084	3089	3084
   query25	704	390	384	384
   query26	1926	174	171	171
   query27	2959	370	356	356
   query28	6667	1859	1831	1831
   query29	1178	635	611	611
   query30	316	151	154	151
   query31	934	743	747	743
   query32	102	62	59	59
   query33	757	255	270	255
   query34	968	474	480	474
   query35	869	652	626	626
   query36	928	815	809	809
   query37	282	76	78	76
   query38	3258	3117	3118	3117
   query39	1411	1366	1372	1366
   query40	306	120	115	115
   query41	63	58	58	58
   query42	111	100	108	100
   query43	442	405	414	405
   query44	1086	721	715	715
   query45	272	254	260	254
   query46	1075	831	786	786
   query47	1619	1533	1572	1533
   query48	418	351	353	351
   query49	1251	380	327	327
   query50	795	379	386	379
   query51	6656	6465	6538	6465
   query52	105	94	94	94
   query53	350	287	289	287
   query54	350	252	251	251
   query55	87	84	86	84
   query56	244	230	228	228
   query57	1086	984	995	984
   query58	253	220	247	220
   query59	2496	2463	2347	2347
   query60	264	255	262	255
   query61	136	115	117	115
   query62	653	434	411	411
   query63	314	282	288	282
   query64	6301	3310	3036	3036
   query65	3028	3038	3001	3001
   query66	1394	335	326	326
   query67	15030	14333	14363	14333
   query68	12957	561	565	561
   query69	724	382	403	382
   query70	1458	1097	1072	1072
   query71	627	272	271	271
   query72	9943	2611	2491	2491
   query73	3303	345	345	345
   query74	7216	6811	6787	6787
   query75	12515	8077	8062	8062
   query76	8050	1100	1169	1100
   query77	902	250	273	250
   query78	10195	9725	9543	9543
   query79	11141	517	506	506
   query80	924	384	379	379
   query81	492	204	212	204
   query82	344	189	196	189
   query83	261	143	142	142
   query84	284	74	76	74
   query85	1192	344	326	326
   query86	372	304	301	301
   query87	3396	3164	3166	3164
   query88	3126	2355	2346	2346
   query89	535	371	363	363
   query90	2335	173	177	173
   query91	162	127	153	127
   query92	64	47	48	47
   query93	4802	542	528	528
   query94	1593	187	196	187
   query95	442	349	347	347
   query96	586	265	263	263
   query97	4456	4278	4286	4278
   query98	226	232	236	232
   query99	1105	775	752	752
   Total cold run time: 321774 ms
   Total hot run time: 186146 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987100666

   TeamCity be ut coverage result:
    Function Coverage: 35.35% (8553/24195) 
    Line Coverage: 27.13% (69490/256127)
    Region Coverage: 26.35% (36052/136831)
    Branch Coverage: 23.20% (18428/79438)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/66d444e9ec35e664ac6e34997d2a39452cd519f0_66d444e9ec35e664ac6e34997d2a39452cd519f0/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987472745

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #32019:
URL: https://github.com/apache/doris/pull/32019


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](lib) use libbase64 to do base64 [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1986069359

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](lib) use libbase64 to do base64 [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1986076620

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987472721

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987101156

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38176 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 66d444e9ec35e664ac6e34997d2a39452cd519f0, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	18533	4151	4139	4139
   q2	2653	178	142	142
   q3	11162	1007	958	958
   q4	5291	967	964	964
   q5	7732	2942	3015	2942
   q6	179	130	126	126
   q7	1311	821	804	804
   q8	9527	2051	2070	2051
   q9	7265	6504	6489	6489
   q10	8263	2557	2543	2543
   q11	441	216	218	216
   q12	805	324	311	311
   q13	17949	2999	2913	2913
   q14	279	255	246	246
   q15	484	443	439	439
   q16	478	389	388	388
   q17	952	870	862	862
   q18	6636	5962	5879	5879
   q19	1589	1525	1527	1525
   q20	542	293	291	291
   q21	7492	3723	3658	3658
   q22	798	318	290	290
   Total cold run time: 110361 ms
   Total hot run time: 38176 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	3999	4000	4050	4000
   q2	319	232	223	223
   q3	2950	2921	2935	2921
   q4	1853	1810	1793	1793
   q5	5248	5249	5232	5232
   q6	210	122	118	118
   q7	2241	1817	1848	1817
   q8	3224	3279	3290	3279
   q9	8558	8548	8536	8536
   q10	6101	3747	3693	3693
   q11	524	434	445	434
   q12	694	552	532	532
   q13	14468	2790	2784	2784
   q14	270	248	257	248
   q15	466	441	431	431
   q16	437	389	399	389
   q17	1682	1672	1665	1665
   q18	7777	7507	7237	7237
   q19	1620	1614	1598	1598
   q20	1970	1738	1731	1731
   q21	4887	4777	4741	4741
   q22	544	468	487	468
   Total cold run time: 70042 ms
   Total hot run time: 53870 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987095595

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](lib) use libbase64 to do base64 [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1986906695

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987104523

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.52 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 66d444e9ec35e664ac6e34997d2a39452cd519f0, data reload: false
   
   query1	0.04	0.04	0.03
   query2	0.06	0.03	0.02
   query3	0.23	0.06	0.06
   query4	1.68	0.10	0.10
   query5	0.48	0.49	0.52
   query6	1.27	0.67	0.66
   query7	0.02	0.02	0.02
   query8	0.04	0.02	0.02
   query9	0.54	0.53	0.53
   query10	0.56	0.56	0.58
   query11	0.13	0.09	0.10
   query12	0.12	0.10	0.10
   query13	0.59	0.57	0.57
   query14	0.75	0.74	0.74
   query15	0.83	0.79	0.80
   query16	0.37	0.38	0.36
   query17	0.99	0.97	0.93
   query18	0.25	0.25	0.25
   query19	1.82	1.70	1.71
   query20	0.01	0.01	0.01
   query21	15.41	0.63	0.64
   query22	3.41	3.68	1.92
   query23	17.26	1.02	0.99
   query24	2.33	0.38	0.38
   query25	0.24	0.05	0.05
   query26	0.20	0.14	0.13
   query27	0.03	0.03	0.03
   query28	12.18	0.85	0.86
   query29	12.56	3.34	3.43
   query30	0.66	0.57	0.57
   query31	2.78	0.34	0.34
   query32	3.39	0.43	0.43
   query33	2.89	2.89	2.90
   query34	15.50	4.34	4.32
   query35	4.31	4.34	4.32
   query36	1.09	1.01	1.02
   query37	0.07	0.05	0.05
   query38	0.05	0.03	0.03
   query39	0.02	0.02	0.02
   query40	0.19	0.13	0.14
   query41	0.08	0.02	0.02
   query42	0.02	0.02	0.02
   query43	0.03	0.02	0.02
   Total cold run time: 105.48 s
   Total hot run time: 30.52 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Function](opt) do base64 opt remove the unless mem alloc [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32019:
URL: https://github.com/apache/doris/pull/32019#issuecomment-1987105110

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit 66d444e9ec35e664ac6e34997d2a39452cd519f0 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       17.3 seconds inserted 10000000 Rows, about 578K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org