You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "XieJiann (via GitHub)" <gi...@apache.org> on 2023/11/21 08:04:01 UTC

[PR] [feature](Nereids) support merge graph in group [doris]

XieJiann opened a new pull request, #27353:
URL: https://github.com/apache/doris/pull/27353

   ## Proposed changes
   
   support merge graph in group
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1820500307

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit c9d63f231bccbc11e73ec3d2d6f0712dd63bc61f, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4879	4679	4665	4665
   q2	371	184	158	158
   q3	2017	1906	1890	1890
   q4	1386	1273	1244	1244
   q5	3957	3931	4032	3931
   q6	249	128	126	126
   q7	1415	874	898	874
   q8	2730	2768	2759	2759
   q9	57214	12846	9610	9610
   q10	10943	3543	3534	3534
   q11	403	247	254	247
   q12	1372	293	293	293
   q13	6289	3824	3807	3807
   q14	316	282	283	282
   q15	593	529	537	529
   q16	667	588	586	586
   q17	1144	948	946	946
   q18	7889	7441	7381	7381
   q19	1723	1683	1700	1683
   q20	562	331	310	310
   q21	7545	3974	3958	3958
   q22	473	373	373	373
   Total cold run time: 114137 ms
   Total hot run time: 49186 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4646	4563	4584	4563
   q2	336	217	240	217
   q3	4008	3993	3993	3993
   q4	2698	2696	2682	2682
   q5	9646	9650	9760	9650
   q6	235	125	125	125
   q7	3025	2500	2472	2472
   q8	4462	4470	4465	4465
   q9	13176	13119	13054	13054
   q10	4122	4209	4211	4209
   q11	765	680	635	635
   q12	982	815	821	815
   q13	4290	3580	3577	3577
   q14	392	350	344	344
   q15	583	522	521	521
   q16	731	678	671	671
   q17	3920	3857	3827	3827
   q18	9555	9135	9099	9099
   q19	1835	1768	1788	1768
   q20	2391	2049	2036	2036
   q21	8910	8774	8628	8628
   q22	871	793	814	793
   Total cold run time: 81579 ms
   Total hot run time: 78144 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1825376301

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1822161499

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit e74a8fc2f3e32cc8ca4f65f361fa69822dcb714a, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4914	4651	4661	4651
   q2	370	145	157	145
   q3	2018	1912	1856	1856
   q4	1384	1299	1259	1259
   q5	3963	3968	4003	3968
   q6	248	128	126	126
   q7	1403	876	914	876
   q8	2752	2764	2754	2754
   q9	9710	9583	9479	9479
   q10	3494	3548	3539	3539
   q11	386	248	252	248
   q12	441	297	292	292
   q13	4556	3788	3790	3788
   q14	318	300	291	291
   q15	584	533	533	533
   q16	661	577	584	577
   q17	1126	984	929	929
   q18	7700	7460	7384	7384
   q19	1692	1693	1647	1647
   q20	590	304	317	304
   q21	4341	3953	3965	3953
   q22	486	373	374	373
   Total cold run time: 53137 ms
   Total hot run time: 48972 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4594	4564	4562	4562
   q2	349	228	256	228
   q3	4014	3991	3993	3991
   q4	2711	2696	2707	2696
   q5	9590	9632	9529	9529
   q6	238	123	121	121
   q7	3024	2471	2470	2470
   q8	4415	4436	4440	4436
   q9	13121	13157	13273	13157
   q10	4083	4197	4222	4197
   q11	770	671	675	671
   q12	978	797	838	797
   q13	4257	3574	3620	3574
   q14	371	347	344	344
   q15	577	525	527	525
   q16	736	658	668	658
   q17	3935	3872	3884	3872
   q18	9475	8967	9101	8967
   q19	1801	1779	1770	1770
   q20	2394	2071	2050	2050
   q21	8845	8624	8670	8624
   q22	903	782	827	782
   Total cold run time: 81181 ms
   Total hot run time: 78021 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1820473576

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.58 seconds
    stream load tsv:          569 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.5 seconds inserted 10000000 Rows, about 350K ops/s
    storage size: 17098939786 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1820415598

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1825376354

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1821228889

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.37 seconds
    stream load tsv:          580 seconds loaded 74807831229 Bytes, about 123 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
    storage size: 17099535335 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1820982847

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "jackwener (via GitHub)" <gi...@apache.org>.
jackwener merged PR #27353:
URL: https://github.com/apache/doris/pull/27353


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1821027550

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1821244477

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.43 seconds
    stream load tsv:          579 seconds loaded 74807831229 Bytes, about 123 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.0 seconds inserted 10000000 Rows, about 357K ops/s
    storage size: 17100670789 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1822168238

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.63 seconds
    stream load tsv:          569 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          33 seconds loaded 861443392 Bytes, about 24 MB/s
    insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
    storage size: 17098536414 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1822120282

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Nereids) support merge graph in group [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27353:
URL: https://github.com/apache/doris/pull/27353#issuecomment-1821223552

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 8b7f80e594630a8a0655d04e5aad14af1f725f40, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4910	4603	4678	4603
   q2	358	156	158	156
   q3	2032	1913	1848	1848
   q4	1388	1255	1266	1255
   q5	3942	3962	4002	3962
   q6	245	129	130	129
   q7	1406	881	899	881
   q8	2727	2780	2753	2753
   q9	9661	9517	9605	9517
   q10	3486	3530	3533	3530
   q11	379	257	247	247
   q12	442	294	290	290
   q13	4539	3827	3802	3802
   q14	313	287	281	281
   q15	593	531	531	531
   q16	661	585	585	585
   q17	1127	934	908	908
   q18	7758	7335	7386	7335
   q19	1665	1665	1666	1665
   q20	545	321	302	302
   q21	4372	3938	3947	3938
   q22	475	373	369	369
   Total cold run time: 53024 ms
   Total hot run time: 48887 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4560	4579	4589	4579
   q2	345	220	261	220
   q3	3998	3994	3989	3989
   q4	2677	2674	2687	2674
   q5	9705	9731	9774	9731
   q6	240	124	122	122
   q7	2997	2486	2447	2447
   q8	4468	4473	4454	4454
   q9	13255	13181	13110	13110
   q10	4086	4208	4212	4208
   q11	746	648	667	648
   q12	977	815	816	815
   q13	4287	3607	3581	3581
   q14	373	353	358	353
   q15	573	529	526	526
   q16	735	659	676	659
   q17	3862	3913	3837	3837
   q18	9585	9035	9031	9031
   q19	1803	1744	1764	1744
   q20	2376	2056	2053	2053
   q21	8785	8564	8544	8544
   q22	902	860	798	798
   Total cold run time: 81335 ms
   Total hot run time: 78123 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org