You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2025/12/24 12:00:45 UTC

[PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

cambyzju opened a new pull request, #59340:
URL: https://github.com/apache/doris/pull/59340

   ### What problem does this PR solve?
   
   **Usage Example:**
   ```
   > select * from test_pk@ro where user_id=9820660;
   +---------+----------+
   | user_id | behavior |
   +---------+----------+
   | 9820660 | hello    |
   +---------+----------+
   ```
   
   
   - When there are overlapping key ranges in Paimon Primary Key table, we cannot use native reader:
   
   ```
   > explain verbose select * from test_pk where user_id=9820660;
   +------------------------------------------------------------------------------------------------------------------------+
   | Explain String(Nereids Planner)                                                                                        |
   +------------------------------------------------------------------------------------------------------------------------+
   | PLAN FRAGMENT 0                                                                                                        |
   |   OUTPUT EXPRS:                                                                                                        |
   |     user_id[#0]                                                                                                        |
   |     behavior[#1]                                                                                                       |
   |   PARTITION: UNPARTITIONED                                                                                             |
   |                                                                                                                        |
   |   HAS_COLO_PLAN_NODE: false                                                                                            |
   |                                                                                                                        |
   |   VRESULT SINK                                                                                                         |
   |      MYSQL_PROTOCAL                                                                                                    |
   |                                                                                                                        |
   |   1:VEXCHANGE                                                                                                          |
   |      offset: 0                                                                                                         |
   |      distribute expr lists:                                                                                            |
   |      tuple ids: 0                                                                                                      |
   |                                                                                                                        |
   | PLAN FRAGMENT 1                                                                                                        |
   |                                                                                                                        |
   |   PARTITION: RANDOM                                                                                                    |
   |                                                                                                                        |
   |   HAS_COLO_PLAN_NODE: false                                                                                            |
   |                                                                                                                        |
   |   STREAM DATA SINK                                                                                                     |
   |     EXCHANGE ID: 01                                                                                                    |
   |     UNPARTITIONED                                                                                                      |
   |                                                                                                                        |
   |   0:VPAIMON_SCAN_NODE(79)                                                                                              |
   |      table: paimon_on_s3.db_paimon.test_pk                                                                             |
   |      predicates: (user_id[#0] = 9820660)                                                                               |
   |      inputSplitNum=1, totalFileSize=0, scanRanges=1                                                                    |
   |      partition=1/0                                                                                                     |
   |      backends:                                                                                                         |
   |        1754986988417                                                                                                   |
   |          /data-0891318b-4e3b-42b5-b85c-3bc8e9b8cc45-0.parquet start: 0 length: 0                                       |
   |      cardinality=1001, numNodes=1                                                                                      |
   |      pushdown agg=NONE                                                                                                 |
   |      paimonNativeReadSplits=0/1                                                                                        |
   |      predicatesFromPaimon:                                                                                             |
   |           Equal(user_id, 9820660)                                                                                      |
   |      PaimonTable: test_pk                                                                                              |
   |      PaimonSplitStats:                                                                                                 |
   |        SplitStat [type=JNI, rowCount=1001, mergedRowCount=NONE, rawFileConvertable=false, hasDeletionVector=false]     |
   |      tuple ids: 0                                                                                                      |
   |                                                                                                                        |
   | Tuples:                                                                                                                |
   | TupleDescriptor{id=0, tbl=test_pk}                                                                                     |
   |   SlotDescriptor{id=0, col=user_id, colUniqueId=0, type=bigint, nullable=true, isAutoIncrement=false, subColPath=null} |
   |   SlotDescriptor{id=1, col=behavior, colUniqueId=1, type=text, nullable=true, isAutoIncrement=false, subColPath=null}  |
   |                                                                                                                        |
   |                                                                                                                        |
   |                                                                                                                        |
   |                                                                                                                        |
   | ========== STATISTICS ==========                                                                                       |
   | planed with unknown column statistics                                                                                  |
   +------------------------------------------------------------------------------------------------------------------------+
   ```
   
   - But if we try to read `ro` table, we could use the Native reader with some data latency:
   
   ```
   > explain verbose select * from test_pk@ro where user_id=9820660;
   +--------------------------------------------------------------------------------------------------------------------------------------+
   | Explain String(Nereids Planner)                                                                                                      |
   +--------------------------------------------------------------------------------------------------------------------------------------+
   | PLAN FRAGMENT 0                                                                                                                      |
   |   OUTPUT EXPRS:                                                                                                                      |
   |     user_id[#0]                                                                                                                      |
   |     behavior[#1]                                                                                                                     |
   |   PARTITION: UNPARTITIONED                                                                                                           |
   |                                                                                                                                      |
   |   HAS_COLO_PLAN_NODE: false                                                                                                          |
   |                                                                                                                                      |
   |   VRESULT SINK                                                                                                                       |
   |      MYSQL_PROTOCAL                                                                                                                  |
   |                                                                                                                                      |
   |   1:VEXCHANGE                                                                                                                        |
   |      offset: 0                                                                                                                       |
   |      distribute expr lists:                                                                                                          |
   |      tuple ids: 0                                                                                                                    |
   |                                                                                                                                      |
   | PLAN FRAGMENT 1                                                                                                                      |
   |                                                                                                                                      |
   |   PARTITION: RANDOM                                                                                                                  |
   |                                                                                                                                      |
   |   HAS_COLO_PLAN_NODE: false                                                                                                          |
   |                                                                                                                                      |
   |   STREAM DATA SINK                                                                                                                   |
   |     EXCHANGE ID: 01                                                                                                                  |
   |     UNPARTITIONED                                                                                                                    |
   |                                                                                                                                      |
   |   0:VPAIMON_SCAN_NODE(79)                                                                                                            |
   |      table: paimon_on_s3.db_paimon.test_pk                                                                                           |
   |      predicates: (user_id[#0] = 9820660)                                                                                             |
   |      inputSplitNum=1, totalFileSize=519490, scanRanges=1                                                                             |
   |      partition=1/0                                                                                                                   |
   |      backends:                                                                                                                       |
   |        1754986988417                                                                                                                 |
   |          s3://warehouse/wh/db_paimon.db/test_pk/bucket-0/data-1ae9ee1e-e6aa-4594-9506-75b6b6ad3820-0.parquet start: 0 length: 519490 |
   |      cardinality=1001, numNodes=1                                                                                                    |
   |      pushdown agg=NONE                                                                                                               |
   |      paimonNativeReadSplits=1/1                                                                                                      |
   |      predicatesFromPaimon:                                                                                                           |
   |           Equal(user_id, 9820660)                                                                                                    |
   |      PaimonTable: test_pk$ro                                                                                                         |
   |      PaimonSplitStats:                                                                                                               |
   |        SplitStat [type=NATIVE, rowCount=1000, mergedRowCount=NONE, rawFileConvertable=true, hasDeletionVector=false]                 |
   |      tuple ids: 0                                                                                                                    |
   |                                                                                                                                      |
   | Tuples:                                                                                                                              |
   | TupleDescriptor{id=0, tbl=test_pk}                                                                                                   |
   |   SlotDescriptor{id=0, col=user_id, colUniqueId=0, type=bigint, nullable=true, isAutoIncrement=false, subColPath=null}               |
   |   SlotDescriptor{id=1, col=behavior, colUniqueId=1, type=text, nullable=true, isAutoIncrement=false, subColPath=null}                |
   |                                                                                                                                      |
   |                                                                                                                                      |
   |                                                                                                                                      |
   |                                                                                                                                      |
   | ========== STATISTICS ==========                                                                                                     |
   | planed with unknown column statistics                                                                                                |
   +--------------------------------------------------------------------------------------------------------------------------------------+
   
   ```
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3692022893

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.2 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit e1a09485c65826631b1f0971441d45ccc106a758, data reload: false
   
   query1	0.05	0.04	0.05
   query2	0.10	0.05	0.05
   query3	0.26	0.08	0.08
   query4	1.61	0.12	0.11
   query5	0.27	0.25	0.24
   query6	1.18	0.65	0.65
   query7	0.03	0.03	0.02
   query8	0.05	0.04	0.04
   query9	0.56	0.51	0.50
   query10	0.55	0.56	0.55
   query11	0.16	0.10	0.10
   query12	0.15	0.12	0.12
   query13	0.61	0.60	0.60
   query14	0.98	0.98	0.97
   query15	0.83	0.80	0.81
   query16	0.39	0.39	0.39
   query17	1.02	1.00	1.03
   query18	0.23	0.22	0.21
   query19	1.93	1.88	1.80
   query20	0.02	0.01	0.01
   query21	15.44	0.29	0.14
   query22	4.66	0.05	0.05
   query23	15.96	0.28	0.11
   query24	1.80	0.58	0.20
   query25	0.08	0.08	0.05
   query26	0.14	0.13	0.13
   query27	0.07	0.06	0.08
   query28	4.53	1.24	1.03
   query29	12.62	3.92	3.26
   query30	0.27	0.14	0.11
   query31	2.81	0.62	0.40
   query32	3.23	0.56	0.44
   query33	2.96	3.00	3.00
   query34	17.03	5.15	4.54
   query35	4.60	4.55	4.59
   query36	0.66	0.50	0.49
   query37	0.11	0.07	0.07
   query38	0.08	0.04	0.04
   query39	0.05	0.04	0.03
   query40	0.16	0.14	0.14
   query41	0.09	0.03	0.03
   query42	0.04	0.02	0.02
   query43	0.05	0.03	0.04
   Total cold run time: 98.42 s
   Total hot run time: 27.2 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691969021

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691551362

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180031 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit e3d143133d46ea81899bff8cebfac0b09985296c, data reload: false
   
   query5	4429	606	475	475
   query6	366	245	259	245
   query7	4229	477	274	274
   query8	309	260	257	257
   query9	8774	2546	2553	2546
   query10	529	394	333	333
   query11	15637	14936	14903	14903
   query12	177	123	116	116
   query13	1276	521	395	395
   query14	5769	3054	2788	2788
   query14_1	2794	2706	2705	2705
   query15	237	202	183	183
   query16	848	457	492	457
   query17	1178	738	620	620
   query18	2446	450	359	359
   query19	236	234	215	215
   query20	123	120	118	118
   query21	227	146	123	123
   query22	3905	3845	3948	3845
   query23	16825	16191	16086	16086
   query23_1	16057	16075	15965	15965
   query24	7353	1683	1241	1241
   query24_1	1250	1287	1280	1280
   query25	597	513	468	468
   query26	1272	273	168	168
   query27	2754	473	307	307
   query28	4488	2131	2106	2106
   query29	861	580	478	478
   query30	318	248	216	216
   query31	811	751	649	649
   query32	78	72	69	69
   query33	533	340	289	289
   query34	912	953	541	541
   query35	767	813	714	714
   query36	858	909	820	820
   query37	138	93	81	81
   query38	2981	3065	3037	3037
   query39	770	757	736	736
   query39_1	701	719	709	709
   query40	228	141	125	125
   query41	69	67	64	64
   query42	114	111	111	111
   query43	439	440	402	402
   query44	1329	753	736	736
   query45	190	190	183	183
   query46	891	990	616	616
   query47	1663	1724	1633	1633
   query48	334	333	250	250
   query49	621	441	357	357
   query50	660	296	217	217
   query51	3827	3828	3786	3786
   query52	110	109	100	100
   query53	322	358	303	303
   query54	287	251	242	242
   query55	77	72	73	72
   query56	294	296	294	294
   query57	1170	1143	1101	1101
   query58	276	255	247	247
   query59	2397	2484	2356	2356
   query60	326	311	296	296
   query61	164	162	165	162
   query62	771	697	687	687
   query63	333	297	313	297
   query64	5128	1295	992	992
   query65	4006	3946	3994	3946
   query66	1512	443	331	331
   query67	15415	15018	14907	14907
   query68	8409	1000	721	721
   query69	516	351	306	306
   query70	1072	996	948	948
   query71	381	329	285	285
   query72	6087	5041	4988	4988
   query73	679	598	311	311
   query74	8933	8983	8813	8813
   query75	3267	3151	2832	2832
   query76	3901	1156	758	758
   query77	626	394	307	307
   query78	9419	9409	8876	8876
   query79	1640	871	619	619
   query80	725	664	550	550
   query81	505	273	233	233
   query82	484	127	101	101
   query83	297	262	243	243
   query84	294	119	102	102
   query85	971	508	452	452
   query86	344	311	270	270
   query87	3180	3218	3088	3088
   query88	4174	2295	2293	2293
   query89	487	426	392	392
   query90	2168	158	160	158
   query91	170	169	147	147
   query92	81	69	68	68
   query93	1443	907	557	557
   query94	477	319	295	295
   query95	581	386	316	316
   query96	600	470	208	208
   query97	2314	2320	2225	2225
   query98	209	192	196	192
   query99	1379	1386	1322	1322
   Total cold run time: 262807 ms
   Total hot run time: 180031 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3689600790

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3889426524

   supported in #60505 60556


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3692197410

   # FE Regression Coverage Report
   Increment line coverage `  0.00% (0/18)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/59340_e1a09485c65826631b1f0971441d45ccc106a758_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/59340_e1a09485c65826631b1f0971441d45ccc106a758_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691463843

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691554494

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.61 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit e3d143133d46ea81899bff8cebfac0b09985296c, data reload: false
   
   query1	0.06	0.05	0.05
   query2	0.10	0.05	0.04
   query3	0.26	0.09	0.09
   query4	1.60	0.11	0.11
   query5	0.28	0.26	0.26
   query6	1.18	0.65	0.63
   query7	0.03	0.03	0.03
   query8	0.05	0.04	0.04
   query9	0.56	0.49	0.51
   query10	0.55	0.56	0.53
   query11	0.16	0.12	0.11
   query12	0.14	0.12	0.11
   query13	0.60	0.60	0.61
   query14	0.99	0.97	0.98
   query15	0.81	0.79	0.81
   query16	0.39	0.38	0.39
   query17	1.05	1.04	1.05
   query18	0.23	0.22	0.22
   query19	1.87	1.81	1.84
   query20	0.02	0.01	0.02
   query21	15.48	0.29	0.14
   query22	6.04	0.05	0.05
   query23	15.94	0.28	0.10
   query24	1.52	0.55	0.56
   query25	0.11	0.10	0.08
   query26	0.14	0.14	0.13
   query27	0.08	0.05	0.06
   query28	4.90	1.22	1.02
   query29	12.65	4.00	3.25
   query30	0.29	0.14	0.12
   query31	2.82	0.62	0.39
   query32	3.23	0.55	0.46
   query33	3.05	3.03	3.10
   query34	16.84	5.17	4.52
   query35	4.57	4.58	4.61
   query36	0.67	0.51	0.48
   query37	0.12	0.07	0.06
   query38	0.07	0.04	0.04
   query39	0.05	0.03	0.03
   query40	0.19	0.14	0.14
   query41	0.08	0.03	0.03
   query42	0.05	0.03	0.02
   query43	0.04	0.04	0.03
   Total cold run time: 99.86 s
   Total hot run time: 27.61 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman closed pull request #59340: [feature](paimon) support read paimon ro (read-optimized) system table
URL: https://github.com/apache/doris/pull/59340


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691668911

   # FE Regression Coverage Report
   Increment line coverage `  0.00% (0/18)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/59340_e3d143133d46ea81899bff8cebfac0b09985296c_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/59340_e3d143133d46ea81899bff8cebfac0b09985296c_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3692002482

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 35864 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit e1a09485c65826631b1f0971441d45ccc106a758, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17611	4219	4044	4044
   q2	2018	361	243	243
   q3	10189	1306	729	729
   q4	10209	810	311	311
   q5	7533	2140	1879	1879
   q6	185	170	139	139
   q7	1007	848	717	717
   q8	9352	1387	1146	1146
   q9	6923	5327	5339	5327
   q10	6850	2380	1994	1994
   q11	545	342	325	325
   q12	661	715	585	585
   q13	17784	3651	2980	2980
   q14	294	294	271	271
   q15	583	516	521	516
   q16	690	684	621	621
   q17	697	833	514	514
   q18	7735	7483	7850	7483
   q19	1218	1058	616	616
   q20	423	369	254	254
   q21	4534	4238	4136	4136
   q22	1147	1070	1034	1034
   Total cold run time: 108188 ms
   Total hot run time: 35864 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4606	4204	4297	4204
   q2	312	440	327	327
   q3	2340	3016	2459	2459
   q4	1397	1918	1456	1456
   q5	4601	4373	4517	4373
   q6	217	169	129	129
   q7	2028	1873	1815	1815
   q8	2853	2473	2472	2472
   q9	7481	7488	7298	7298
   q10	2927	3085	2673	2673
   q11	601	518	509	509
   q12	664	689	576	576
   q13	3280	3615	3018	3018
   q14	265	287	256	256
   q15	535	492	488	488
   q16	602	646	605	605
   q17	1119	1405	1332	1332
   q18	7376	7055	7210	7055
   q19	827	774	792	774
   q20	1914	1961	1851	1851
   q21	4602	4391	4194	4194
   q22	1049	1022	986	986
   Total cold run time: 51596 ms
   Total hot run time: 48850 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3691541291

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 35565 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit e3d143133d46ea81899bff8cebfac0b09985296c, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17652	4208	4051	4051
   q2	2023	353	237	237
   q3	10178	1363	780	780
   q4	10236	944	324	324
   q5	7530	2170	2087	2087
   q6	192	175	142	142
   q7	1023	846	722	722
   q8	9380	1474	1212	1212
   q9	7188	5395	5365	5365
   q10	6846	2422	1993	1993
   q11	541	341	314	314
   q12	689	753	564	564
   q13	17785	3737	3068	3068
   q14	289	312	267	267
   q15	606	520	509	509
   q16	733	686	646	646
   q17	722	866	489	489
   q18	7514	7105	7122	7105
   q19	1102	989	610	610
   q20	409	377	248	248
   q21	4366	3896	3931	3896
   q22	1073	1008	936	936
   Total cold run time: 108077 ms
   Total hot run time: 35565 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4139	4064	4025	4025
   q2	348	395	320	320
   q3	2159	2669	2319	2319
   q4	1359	1774	1292	1292
   q5	4324	4854	4731	4731
   q6	231	180	142	142
   q7	2117	1960	1912	1912
   q8	3005	2632	2586	2586
   q9	7835	7704	7384	7384
   q10	3115	3325	2863	2863
   q11	620	535	511	511
   q12	687	755	604	604
   q13	3589	3981	3335	3335
   q14	301	302	301	301
   q15	572	519	505	505
   q16	656	702	655	655
   q17	1223	1472	1514	1472
   q18	8080	7656	7583	7583
   q19	925	934	944	934
   q20	2015	2055	1922	1922
   q21	4941	4654	4260	4260
   q22	1076	994	980	980
   Total cold run time: 53317 ms
   Total hot run time: 50636 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59340:
URL: https://github.com/apache/doris/pull/59340#issuecomment-3692014613

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180298 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit e1a09485c65826631b1f0971441d45ccc106a758, data reload: false
   
   query5	5180	571	444	444
   query6	326	229	213	213
   query7	4222	454	275	275
   query8	299	258	230	230
   query9	8743	2538	2516	2516
   query10	545	352	321	321
   query11	15377	14964	14807	14807
   query12	173	123	122	122
   query13	1272	473	400	400
   query14	6138	2952	2686	2686
   query14_1	2602	2572	2578	2572
   query15	215	191	173	173
   query16	799	472	443	443
   query17	1078	693	565	565
   query18	2457	429	329	329
   query19	226	226	200	200
   query20	121	116	112	112
   query21	225	139	124	124
   query22	4086	4063	4123	4063
   query23	16617	16377	16227	16227
   query23_1	16158	16438	16449	16438
   query24	7367	1669	1256	1256
   query24_1	1237	1237	1216	1216
   query25	572	498	441	441
   query26	1247	271	167	167
   query27	2734	471	304	304
   query28	4483	2106	2131	2106
   query29	827	566	481	481
   query30	318	245	221	221
   query31	815	701	633	633
   query32	79	76	68	68
   query33	570	336	319	319
   query34	887	891	538	538
   query35	780	801	721	721
   query36	849	935	854	854
   query37	130	91	81	81
   query38	3012	3074	2962	2962
   query39	767	750	763	750
   query39_1	711	742	704	704
   query40	230	149	129	129
   query41	76	70	66	66
   query42	112	107	106	106
   query43	418	446	405	405
   query44	1321	753	741	741
   query45	191	191	185	185
   query46	884	971	608	608
   query47	1706	1778	1670	1670
   query48	327	326	245	245
   query49	644	450	364	364
   query50	661	300	227	227
   query51	3809	3861	3807	3807
   query52	114	109	99	99
   query53	321	357	299	299
   query54	300	270	259	259
   query55	76	78	74	74
   query56	297	310	321	310
   query57	1193	1174	1104	1104
   query58	281	256	266	256
   query59	2463	2462	2473	2462
   query60	365	308	296	296
   query61	166	150	163	150
   query62	710	711	640	640
   query63	326	286	295	286
   query64	4948	1267	978	978
   query65	4028	3926	3967	3926
   query66	1448	471	323	323
   query67	15355	15255	14928	14928
   query68	8428	993	698	698
   query69	503	344	318	318
   query70	1089	1030	975	975
   query71	363	311	287	287
   query72	6118	4984	5141	4984
   query73	705	651	306	306
   query74	8959	8917	8739	8739
   query75	3195	3164	2768	2768
   query76	3870	1140	737	737
   query77	517	394	292	292
   query78	9512	9642	8901	8901
   query79	1755	836	606	606
   query80	709	653	558	558
   query81	514	271	233	233
   query82	195	130	104	104
   query83	260	258	243	243
   query84	258	122	110	110
   query85	932	500	449	449
   query86	378	313	286	286
   query87	3205	3154	3225	3154
   query88	3244	2276	2264	2264
   query89	457	435	397	397
   query90	2165	159	159	159
   query91	174	167	143	143
   query92	75	68	65	65
   query93	1388	885	550	550
   query94	464	281	276	276
   query95	567	373	302	302
   query96	588	463	208	208
   query97	2241	2340	2255	2255
   query98	219	194	195	194
   query99	1325	1405	1291	1291
   Total cold run time: 261201 ms
   Total hot run time: 180298 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](paimon) support read paimon ro (read-optimized) system table [doris]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #59340:
URL: https://github.com/apache/doris/pull/59340#discussion_r2647100927


##########
fe/fe-core/src/main/java/org/apache/doris/analysis/TableScanParams.java:
##########
@@ -29,24 +29,32 @@ public class TableScanParams {
     public static final String INCREMENTAL_READ = "incr";
     public static final String BRANCH = "branch";
     public static final String TAG = "tag";
+    // for paimon read-optimized system table
+    public static final String RO = "ro";
     private static final ImmutableSet<String> VALID_PARAM_TYPES = ImmutableSet.of(
             INCREMENTAL_READ,
             BRANCH,
-            TAG);
+            TAG,
+            RO);
 
     private final String paramType;
-    // There are two ways to pass parameters to a function.
+    // There are three ways to pass parameters to a function.
     // - One is in map form, where the data is stored in `mapParams`.
     //   such as: @func_name('param1'='value1', 'param2'='value2', 'param3'='value3')
-    // - The other is in list form, where the data is stored in `listParams`.
+    // - Another other one is in list form, where the data is stored in `listParams`.
     //   such as: `listParams` is used for @func_name('value1', 'value2', 'value3')
+    // - The third one is in short form, now only used for paimon 'ro' table.
+    //   such as: @ro
     private final Map<String, String> mapParams;
     private final List<String> listParams;
 
     private void validate() {
         if (!VALID_PARAM_TYPES.contains(paramType)) {
             throw new IllegalArgumentException("Invalid param type: " + paramType);
         }
+        if (isRo() && (!mapParams.isEmpty() || !listParams.isEmpty())) {
+            throw new IllegalArgumentException("Please use '@ro' for Paimon read-optimized system table");

Review Comment:
   The error message says "Please use '@ro' for Paimon read-optimized system table", but this validation is triggered when RO is used WITH parameters (like @ro('param'='value')). The message could be clearer by specifying that RO should be used without parameters, for example: "The '@ro' parameter for Paimon read-optimized tables should be used without arguments".
   ```suggestion
               throw new IllegalArgumentException("The '@ro' parameter for Paimon read-optimized tables must be used without arguments");
   ```



##########
fe/fe-core/src/main/java/org/apache/doris/analysis/TableScanParams.java:
##########
@@ -29,24 +29,32 @@ public class TableScanParams {
     public static final String INCREMENTAL_READ = "incr";
     public static final String BRANCH = "branch";
     public static final String TAG = "tag";
+    // for paimon read-optimized system table
+    public static final String RO = "ro";
     private static final ImmutableSet<String> VALID_PARAM_TYPES = ImmutableSet.of(
             INCREMENTAL_READ,
             BRANCH,
-            TAG);
+            TAG,
+            RO);
 
     private final String paramType;
-    // There are two ways to pass parameters to a function.
+    // There are three ways to pass parameters to a function.
     // - One is in map form, where the data is stored in `mapParams`.
     //   such as: @func_name('param1'='value1', 'param2'='value2', 'param3'='value3')
-    // - The other is in list form, where the data is stored in `listParams`.
+    // - Another other one is in list form, where the data is stored in `listParams`.

Review Comment:
   The phrase "Another other one" is grammatically incorrect. It should be either "Another one" or "The other one" to maintain grammatical clarity.
   ```suggestion
       // - Another one is in list form, where the data is stored in `listParams`.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org