You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2024/01/31 14:20:02 UTC

[PR] [opt](catalog) cache the converted properties [doris]

morningman opened a new pull request, #30668:
URL: https://github.com/apache/doris/pull/30668

   ## Proposed changes
   
   convert properties may be a heavy operation, so we cache the result.
   
   ```
           at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:480)
           - waiting to lock <0x00000005e99cca00> (a sun.misc.URLClassPath)
           at sun.misc.URLClassPath.findResource(URLClassPath.java:225)
           at java.net.URLClassLoader$2.run(URLClassLoader.java:577)
           at java.net.URLClassLoader$2.run(URLClassLoader.java:575)
           at java.security.AccessController.doPrivileged(Native Method)
           at java.net.URLClassLoader.findResource(URLClassLoader.java:574)
           at java.lang.ClassLoader.getResource(ClassLoader.java:1089)
           at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2839)
           at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3113)
           at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3072)
           at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3045)
           at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2923)
           - locked <0x000000065cfe78a8> (a org.apache.hadoop.hive.conf.HiveConf)
           at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2905)
           - locked <0x000000065cfe78a8> (a org.apache.hadoop.hive.conf.HiveConf)
           at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2968)
           at org.apache.hadoop.hive.conf.HiveConf.getProperties(HiveConf.java:5134)
           at org.apache.hadoop.hive.conf.HiveConf.getAllProperties(HiveConf.java:5130)
           at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5147)
           at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5102)
           at org.apache.doris.datasource.property.PropertyConverter.getPropertiesFromDLFConf(PropertyConverter.java:377)
           at org.apache.doris.datasource.property.PropertyConverter.convertToDLFProperties(PropertyConverter.java:365)
           at org.apache.doris.datasource.property.PropertyConverter.convertToMetaProperties(PropertyConverter.java:89)
           at org.apache.doris.datasource.ExternalCatalog.getProperties(ExternalCatalog.java:407)
           at org.apache.doris.datasource.hive.HiveMetaStoreCache.getFileCache(HiveMetaStoreCache.java:378)
   ```
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925565947

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7 with default session variables
   Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
   Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
   Insert into select:       13.7 seconds inserted 10000000 Rows, about 729K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919430718

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36599 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17645	4401	4307	4307
   q2	2035	140	131	131
   q3	10598	898	891	891
   q4	4685	765	751	751
   q5	7660	2888	2718	2718
   q6	184	117	117	117
   q7	1127	712	698	698
   q8	9253	1998	1985	1985
   q9	7163	6340	6276	6276
   q10	8115	2405	2419	2405
   q11	415	192	208	192
   q12	735	269	286	269
   q13	18013	3291	3266	3266
   q14	275	244	246	244
   q15	537	488	483	483
   q16	458	401	425	401
   q17	921	556	500	500
   q18	6756	6092	6003	6003
   q19	1548	1379	1348	1348
   q20	570	328	338	328
   q21	6743	2985	2990	2985
   q22	788	303	301	301
   Total cold run time: 106224 ms
   Total hot run time: 36599 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4359	4359	4396	4359
   q2	325	237	234	234
   q3	3034	2829	2817	2817
   q4	1853	1610	1665	1610
   q5	5211	5224	5226	5224
   q6	187	111	115	111
   q7	2090	1796	1821	1796
   q8	3141	3223	3246	3223
   q9	8304	8291	8260	8260
   q10	5845	3570	3554	3554
   q11	561	451	450	450
   q12	725	572	583	572
   q13	12557	3064	3070	3064
   q14	274	243	265	243
   q15	523	486	489	486
   q16	513	480	459	459
   q17	1821	1675	1702	1675
   q18	8034	7722	7564	7564
   q19	9263	1542	1556	1542
   q20	2140	1898	1890	1890
   q21	4825	4654	4531	4531
   q22	540	463	459	459
   Total cold run time: 76125 ms
   Total hot run time: 54123 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925565250

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.53 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
   
   query1	0.04	0.02	0.03
   query2	0.06	0.02	0.02
   query3	0.24	0.06	0.06
   query4	1.64	0.10	0.09
   query5	0.52	0.51	0.53
   query6	1.21	0.64	0.63
   query7	0.02	0.01	0.01
   query8	0.04	0.02	0.02
   query9	0.57	0.52	0.48
   query10	0.57	0.57	0.55
   query11	0.12	0.08	0.08
   query12	0.11	0.09	0.09
   query13	0.60	0.60	0.61
   query14	0.79	0.81	0.80
   query15	0.79	0.78	0.78
   query16	0.38	0.38	0.38
   query17	1.02	1.03	1.03
   query18	0.25	0.23	0.24
   query19	1.81	1.81	1.78
   query20	0.01	0.01	0.01
   query21	15.40	0.57	0.58
   query22	2.55	3.00	2.05
   query23	17.33	0.77	0.72
   query24	2.69	0.28	0.68
   query25	0.40	0.19	0.08
   query26	0.36	0.15	0.14
   query27	0.04	0.05	0.05
   query28	12.22	0.82	0.85
   query29	12.55	3.20	3.20
   query30	0.55	0.57	0.53
   query31	2.78	0.35	0.36
   query32	3.36	0.49	0.48
   query33	3.21	3.26	3.21
   query34	15.84	4.31	4.32
   query35	4.40	4.36	4.27
   query36	1.13	1.06	1.06
   query37	0.06	0.05	0.04
   query38	0.04	0.02	0.02
   query39	0.03	0.01	0.01
   query40	0.16	0.13	0.15
   query41	0.07	0.02	0.01
   query42	0.02	0.01	0.01
   query43	0.03	0.02	0.02
   Total cold run time: 106.01 s
   Total hot run time: 30.53 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925564307

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 174449 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
   
   query1	935	349	333	333
   query2	6544	2071	1909	1909
   query3	6700	206	200	200
   query4	31690	22036	22022	22022
   query5	4270	356	361	356
   query6	258	172	177	172
   query7	4613	285	277	277
   query8	252	175	194	175
   query9	8863	2313	2296	2296
   query10	409	213	214	213
   query11	18649	15467	15326	15326
   query12	124	76	78	76
   query13	1612	407	406	406
   query14	9412	6886	6939	6886
   query15	289	182	182	182
   query16	8160	253	253	253
   query17	2043	532	490	490
   query18	2104	271	282	271
   query19	368	142	137	137
   query20	81	73	79	73
   query21	199	120	123	120
   query22	4901	4645	4680	4645
   query23	31056	29983	29946	29946
   query24	6430	2754	2753	2753
   query25	545	347	343	343
   query26	702	147	142	142
   query27	2127	303	304	303
   query28	4811	1884	1854	1854
   query29	882	614	609	609
   query30	282	130	142	130
   query31	931	711	696	696
   query32	89	59	49	49
   query33	464	220	217	217
   query34	815	451	468	451
   query35	846	780	745	745
   query36	992	914	946	914
   query37	98	59	62	59
   query38	3216	3120	3108	3108
   query39	1290	1258	1220	1220
   query40	181	93	91	91
   query41	37	36	34	34
   query42	95	87	90	87
   query43	512	472	463	463
   query44	1022	699	699	699
   query45	187	179	176	176
   query46	1033	649	647	647
   query47	1515	1531	1498	1498
   query48	428	353	361	353
   query49	1044	289	286	286
   query50	762	369	369	369
   query51	5272	5177	5094	5094
   query52	93	87	81	81
   query53	337	275	259	259
   query54	258	220	220	220
   query55	79	77	86	77
   query56	214	195	196	195
   query57	949	892	902	892
   query58	202	177	179	177
   query59	2464	2449	2375	2375
   query60	237	214	220	214
   query61	87	86	84	84
   query62	627	356	346	346
   query63	286	275	274	274
   query64	4648	3537	3715	3537
   query65	3273	3229	3201	3201
   query66	830	312	303	303
   query67	14209	14266	14129	14129
   query68	4398	534	525	525
   query69	471	337	345	337
   query70	1214	1229	1224	1224
   query71	315	250	252	250
   query72	6090	2868	2685	2685
   query73	698	316	317	316
   query74	6571	6342	6234	6234
   query75	3060	2321	2279	2279
   query76	2593	986	998	986
   query77	343	230	223	223
   query78	9257	8786	8589	8589
   query79	2270	495	490	490
   query80	1529	351	347	347
   query81	534	194	198	194
   query82	645	79	81	79
   query83	240	120	121	120
   query84	237	80	78	78
   query85	2152	364	337	337
   query86	494	277	308	277
   query87	3392	3214	3166	3166
   query88	3705	2307	2320	2307
   query89	437	356	367	356
   query90	1944	168	163	163
   query91	168	146	124	124
   query92	57	42	44	42
   query93	2644	505	458	458
   query94	1302	176	169	169
   query95	468	360	354	354
   query96	596	264	267	264
   query97	4240	4104	4114	4104
   query98	219	197	197	197
   query99	1192	696	657	657
   Total cold run time: 272408 ms
   Total hot run time: 174449 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919469761

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d with default session variables
   Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
   Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925555758

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921737735

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919199449

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926548736

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #30668:
URL: https://github.com/apache/doris/pull/30668


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919370516

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921608657

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926564294

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925562208

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 37589 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17886	4502	4443	4443
   q2	2044	145	136	136
   q3	10590	935	952	935
   q4	5148	756	746	746
   q5	7739	3000	2918	2918
   q6	183	121	122	121
   q7	1117	750	733	733
   q8	9312	2025	2010	2010
   q9	7382	6511	6569	6511
   q10	8299	2615	2696	2615
   q11	415	222	207	207
   q12	763	300	288	288
   q13	18704	3355	3383	3355
   q14	287	256	269	256
   q15	547	509	500	500
   q16	1297	417	418	417
   q17	1734	561	540	540
   q18	6840	5988	5895	5895
   q19	1584	1499	1375	1375
   q20	629	356	318	318
   q21	6800	2972	3056	2972
   q22	800	304	298	298
   Total cold run time: 110100 ms
   Total hot run time: 37589 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4538	4345	4494	4345
   q2	330	240	237	237
   q3	2982	2867	2801	2801
   q4	1821	1629	1665	1629
   q5	5171	5262	5231	5231
   q6	194	115	117	115
   q7	2145	1721	1762	1721
   q8	3087	3221	3275	3221
   q9	8320	8260	8282	8260
   q10	5774	3507	3573	3507
   q11	539	467	456	456
   q12	767	579	559	559
   q13	14665	3072	3089	3072
   q14	274	251	262	251
   q15	528	491	492	491
   q16	537	471	470	470
   q17	1864	1677	1691	1677
   q18	7987	7816	7479	7479
   q19	8611	1512	1544	1512
   q20	2146	1906	1898	1898
   q21	5085	4617	4543	4543
   q22	565	464	442	442
   Total cold run time: 77930 ms
   Total hot run time: 53917 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921696264

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36977 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17702	4665	4421	4421
   q2	2042	135	131	131
   q3	10678	920	896	896
   q4	4676	711	715	711
   q5	7672	2847	2843	2843
   q6	181	117	117	117
   q7	1134	725	717	717
   q8	9310	2014	2026	2014
   q9	7227	6327	6304	6304
   q10	8084	2448	2435	2435
   q11	409	201	209	201
   q12	773	269	265	265
   q13	18018	3276	3292	3276
   q14	271	249	242	242
   q15	530	480	495	480
   q16	465	413	394	394
   q17	934	576	569	569
   q18	6849	5899	5830	5830
   q19	1576	1373	1373	1373
   q20	577	326	334	326
   q21	6876	3177	3127	3127
   q22	803	306	305	305
   Total cold run time: 106787 ms
   Total hot run time: 36977 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4428	4489	4372	4372
   q2	328	239	245	239
   q3	2943	2817	2807	2807
   q4	1821	1740	1617	1617
   q5	5238	5192	5251	5192
   q6	188	113	113	113
   q7	2170	1785	1750	1750
   q8	3139	3221	3276	3221
   q9	8274	8264	8364	8264
   q10	5754	3581	3523	3523
   q11	537	453	455	453
   q12	750	574	590	574
   q13	13522	3086	3071	3071
   q14	272	246	257	246
   q15	533	494	492	492
   q16	497	471	471	471
   q17	1804	1700	1690	1690
   q18	8027	7566	7585	7566
   q19	9281	1532	1482	1482
   q20	2160	1903	1894	1894
   q21	6312	4544	4537	4537
   q22	541	468	430	430
   Total cold run time: 78519 ms
   Total hot run time: 54004 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921732039

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.57 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
   
   query1	0.04	0.03	0.03
   query2	0.06	0.02	0.02
   query3	0.23	0.06	0.06
   query4	1.68	0.10	0.10
   query5	0.53	0.51	0.51
   query6	1.17	0.63	0.63
   query7	0.02	0.01	0.01
   query8	0.04	0.02	0.03
   query9	0.55	0.50	0.50
   query10	0.54	0.54	0.56
   query11	0.12	0.08	0.08
   query12	0.11	0.08	0.09
   query13	0.60	0.61	0.61
   query14	0.79	0.79	0.78
   query15	0.78	0.77	0.77
   query16	0.38	0.37	0.40
   query17	0.99	0.98	1.01
   query18	0.22	0.26	0.25
   query19	1.79	1.74	1.76
   query20	0.02	0.01	0.01
   query21	15.41	0.58	0.57
   query22	2.57	2.54	1.66
   query23	16.87	0.89	0.80
   query24	2.69	1.59	0.84
   query25	0.26	0.31	0.08
   query26	0.60	0.13	0.14
   query27	0.05	0.05	0.05
   query28	12.38	0.84	0.83
   query29	12.66	3.10	3.22
   query30	0.62	0.54	0.53
   query31	2.79	0.33	0.36
   query32	3.37	0.47	0.49
   query33	3.23	3.22	3.22
   query34	15.86	4.32	4.28
   query35	4.29	4.27	4.30
   query36	1.12	1.06	1.07
   query37	0.07	0.05	0.05
   query38	0.03	0.03	0.03
   query39	0.02	0.01	0.02
   query40	0.16	0.14	0.12
   query41	0.07	0.01	0.01
   query42	0.03	0.02	0.03
   query43	0.03	0.02	0.02
   Total cold run time: 105.84 s
   Total hot run time: 30.57 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921720389

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 173261 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
   
   query1	933	333	323	323
   query2	6580	1927	1899	1899
   query3	6696	201	195	195
   query4	31961	21927	21955	21927
   query5	4456	356	412	356
   query6	249	158	153	153
   query7	4597	265	261	261
   query8	237	180	170	170
   query9	9171	2300	2267	2267
   query10	409	195	215	195
   query11	18706	15471	15355	15355
   query12	123	69	64	64
   query13	1621	367	377	367
   query14	9955	6914	7461	6914
   query15	233	182	184	182
   query16	7324	264	251	251
   query17	1865	485	461	461
   query18	1933	256	246	246
   query19	199	127	134	127
   query20	68	68	71	68
   query21	196	147	124	124
   query22	4962	4637	4566	4566
   query23	31639	30184	30223	30184
   query24	7315	2744	2726	2726
   query25	526	309	316	309
   query26	731	138	138	138
   query27	2259	279	277	277
   query28	5286	1848	1836	1836
   query29	929	617	606	606
   query30	284	133	138	133
   query31	924	734	722	722
   query32	95	58	56	56
   query33	457	213	201	201
   query34	814	442	466	442
   query35	824	755	742	742
   query36	950	920	921	920
   query37	94	55	57	55
   query38	3267	3131	3159	3131
   query39	1310	1264	1235	1235
   query40	193	87	88	87
   query41	39	36	40	36
   query42	91	82	79	79
   query43	528	476	507	476
   query44	1039	705	704	704
   query45	192	178	174	174
   query46	1028	619	612	612
   query47	1531	1557	1489	1489
   query48	385	305	311	305
   query49	1122	285	284	284
   query50	694	304	299	299
   query51	5287	5152	5124	5124
   query52	93	76	80	76
   query53	319	258	263	258
   query54	225	181	185	181
   query55	79	74	78	74
   query56	175	165	154	154
   query57	975	894	911	894
   query58	179	158	162	158
   query59	2442	2298	2428	2298
   query60	204	171	175	171
   query61	84	87	84	84
   query62	581	357	340	340
   query63	281	255	260	255
   query64	4681	3691	3335	3335
   query65	3250	3228	3220	3220
   query66	959	318	318	318
   query67	14608	13974	13841	13841
   query68	4025	497	502	497
   query69	425	303	309	303
   query70	1285	1218	1183	1183
   query71	299	209	212	209
   query72	5540	3074	2846	2846
   query73	689	310	315	310
   query74	6672	6223	6247	6223
   query75	2942	2274	2264	2264
   query76	2561	936	1017	936
   query77	357	230	220	220
   query78	9315	8596	8511	8511
   query79	3068	481	482	481
   query80	2034	325	309	309
   query81	495	202	207	202
   query82	763	80	81	80
   query83	244	122	116	116
   query84	280	72	73	72
   query85	2006	345	336	336
   query86	478	322	285	285
   query87	3462	3271	3211	3211
   query88	4127	2169	2152	2152
   query89	445	360	352	352
   query90	1912	185	184	184
   query91	161	116	116	116
   query92	53	44	40	40
   query93	5362	452	443	443
   query94	1210	157	164	157
   query95	504	470	444	444
   query96	630	310	309	309
   query97	4229	4103	4115	4103
   query98	207	189	184	184
   query99	1177	673	673	673
   Total cold run time: 278399 ms
   Total hot run time: 173261 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926315121

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919452642

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 175700 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
   
   query1	925	336	329	329
   query2	6574	2026	1901	1901
   query3	6700	202	200	200
   query4	31945	22209	22190	22190
   query5	4452	432	352	352
   query6	250	161	155	155
   query7	4599	262	271	262
   query8	245	174	175	174
   query9	9270	2270	2270	2270
   query10	423	210	217	210
   query11	18706	15470	15380	15380
   query12	121	70	67	67
   query13	1628	403	385	385
   query14	10221	7551	7350	7350
   query15	247	177	181	177
   query16	7528	267	251	251
   query17	1984	501	459	459
   query18	1968	258	249	249
   query19	328	138	135	135
   query20	75	75	68	68
   query21	206	138	125	125
   query22	5013	4830	4771	4771
   query23	31739	30342	30379	30342
   query24	10217	2755	2781	2755
   query25	503	314	310	310
   query26	709	138	139	138
   query27	2095	286	300	286
   query28	5538	1853	1835	1835
   query29	901	624	611	611
   query30	282	134	139	134
   query31	900	702	710	702
   query32	88	52	48	48
   query33	606	206	212	206
   query34	815	448	461	448
   query35	861	765	738	738
   query36	1360	1178	1212	1178
   query37	90	58	57	57
   query38	3320	3183	3146	3146
   query39	1310	1265	1240	1240
   query40	191	90	84	84
   query41	38	35	36	35
   query42	88	81	83	81
   query43	513	493	497	493
   query44	1032	683	689	683
   query45	194	184	177	177
   query46	1059	640	622	622
   query47	1555	1406	1507	1406
   query48	419	301	307	301
   query49	1097	289	272	272
   query50	683	317	315	315
   query51	5273	5200	5175	5175
   query52	102	73	79	73
   query53	320	264	266	264
   query54	276	187	182	182
   query55	82	73	78	73
   query56	179	162	159	159
   query57	965	909	904	904
   query58	182	163	158	158
   query59	2553	2277	2380	2277
   query60	202	176	187	176
   query61	83	83	83	83
   query62	602	361	361	361
   query63	276	260	268	260
   query64	4617	3711	3468	3468
   query65	3240	3236	3217	3217
   query66	940	324	318	318
   query67	14353	14199	14236	14199
   query68	4383	498	499	498
   query69	458	310	315	310
   query70	1591	1568	1539	1539
   query71	282	214	212	212
   query72	5867	3134	2845	2845
   query73	677	317	318	317
   query74	6639	6179	6270	6179
   query75	3050	2252	2313	2252
   query76	3201	982	958	958
   query77	377	236	229	229
   query78	9219	8676	8553	8553
   query79	2802	502	502	502
   query80	2122	325	319	319
   query81	541	194	204	194
   query82	709	85	86	85
   query83	334	114	119	114
   query84	275	82	76	76
   query85	1692	330	339	330
   query86	517	403	376	376
   query87	3418	3223	3273	3223
   query88	3773	2156	2174	2156
   query89	420	366	359	359
   query90	1945	186	180	180
   query91	148	114	120	114
   query92	52	45	42	42
   query93	2462	429	418	418
   query94	1420	159	161	159
   query95	503	465	455	455
   query96	618	315	308	308
   query97	4235	4114	4116	4114
   query98	220	198	176	176
   query99	1142	679	658	658
   Total cold run time: 280948 ms
   Total hot run time: 175700 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](catalog) cache the converted properties [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919464100

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.77 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
   
   query1	0.04	0.03	0.03
   query2	0.05	0.02	0.02
   query3	0.24	0.06	0.06
   query4	1.69	0.10	0.10
   query5	0.52	0.52	0.51
   query6	1.20	0.67	0.64
   query7	0.02	0.02	0.01
   query8	0.03	0.02	0.02
   query9	0.57	0.52	0.50
   query10	0.56	0.55	0.56
   query11	0.12	0.08	0.08
   query12	0.12	0.10	0.09
   query13	0.61	0.61	0.61
   query14	0.78	0.80	0.81
   query15	0.80	0.78	0.77
   query16	0.37	0.39	0.39
   query17	0.99	1.01	1.01
   query18	0.24	0.25	0.26
   query19	1.87	1.70	1.77
   query20	0.01	0.01	0.01
   query21	15.40	0.58	0.57
   query22	2.67	2.32	1.81
   query23	17.11	0.88	0.85
   query24	2.51	0.85	1.11
   query25	0.26	0.26	0.07
   query26	0.51	0.15	0.15
   query27	0.06	0.04	0.04
   query28	11.40	0.86	0.84
   query29	12.55	3.08	3.12
   query30	0.63	0.47	0.51
   query31	2.78	0.35	0.34
   query32	3.37	0.48	0.49
   query33	3.22	3.23	3.22
   query34	15.74	4.27	4.29
   query35	4.29	4.33	4.27
   query36	1.11	1.06	1.07
   query37	0.07	0.05	0.05
   query38	0.04	0.03	0.02
   query39	0.02	0.02	0.02
   query40	0.16	0.12	0.12
   query41	0.06	0.02	0.01
   query42	0.03	0.01	0.02
   query43	0.02	0.02	0.02
   Total cold run time: 104.84 s
   Total hot run time: 30.77 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org