You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xylaaaaa (via GitHub)" <gi...@apache.org> on 2026/10/21 08:58:57 UTC
[PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
xylaaaaa opened a new pull request, #69392:
URL: https://github.com/apache/doris/pull/69392
### What problem does this PR solve?
Issue Number: None
Related PR: #69082
Problem Summary: DELETE, UPDATE and MERGE on an Iceberg table whose location FE normalizes for BE
(oss://, cos://, cosn://, obs://, bos://, gs://, s3a://, s3n://, and Azure abfs/abfss/wasb/wasbs
except OneLake) record the new delete files in Iceberg metadata with the internal s3:// address
instead of the table's own scheme. Both v2 position delete files (delete_pos_*.parquet) and v3
deletion vector files (delete_dv_*.puffin) are affected; data files are not.
Reproduce on an Iceberg table located at `oss://bucket/...`:
```sql
CREATE TABLE t (id INT, v STRING) PROPERTIES ('format-version' = '2');
INSERT INTO t VALUES (1, 'a'), (2, 'b'), (3, 'c');
DELETE FROM t WHERE id = 1;
SELECT file_path FROM t$delete_files; -- s3://bucket/.../data/delete_pos_xxx.parquet
SELECT file_path FROM t$data_files; -- oss://bucket/.../data/...
```
Doris itself still reads the table, because it normalizes every path it reads anyway, but other
engines resolve the file system by scheme and fail: Spark or Trino reading the table, or running
expire_snapshots / remove_orphan_files, look the delete file up as s3 (e.g. OSSFileIO rejects the
s3 scheme, HadoopFileIO usually has no file system for it, ResolvingFileIO goes to AWS S3,
ADLSFileIO rejects it as an invalid ADLS URI).
Cause: data files are written under write_path (the s3:// form BE can address) and committed
under original_write_path (VIcebergPartitionWriter). VIcebergDeleteSink had no such split: it
generates delete file paths under output_path and reports exactly that path for the commit
(VIcebergDeleteFileWriter sets file_path to its output path, and the puffin path is reported
as written).
Fix: apply the same split to delete files. VIcebergDeleteSink still writes under output_path but
commits under table_location, the raw data location FE already sends, e.g. written to
s3://b/db/t/data/delete_x.parquet and committed as oss://b/db/t/data/delete_x.parquet. Every FE
version sets table_location to the raw form of output_path's directory (delete sink and merge
sink alike), so no protocol change is needed. That contract is now written on table_location in
DataSinks.thrift (comments only) and asserted by an FE unit test, since breaking it would record
delete files that do not exist without any error.
Delete files that an older version already committed with s3:// keep that address until they
are rewritten.
### Release note
Fix Iceberg DELETE/UPDATE/MERGE recording delete files with an s3:// path instead of the table's
own location (e.g. oss:// or abfss://), which made other engines fail to read the table.
### Check List (For Author)
- Test: Unit Test (VIcebergDeleteSinkTest.TestCommitPathUsesTableLocation,
IcebergWritePlanProviderTest.planWriteDeleteAndMergeSinksKeepTableLocationInTheOutputDirectory;
not run locally, CI runs them)
- Behavior changed: Yes. Delete files are committed under table_location.
- Does this need documentation: No
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6409231556
<details>
<summary>TPC-DS: <b>Total hot run time: 150691 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 52c04ac75925f60e70c8415f91035b7efb97f2aa, data reload: false
query5 4322 595 446 446
query6 426 210 189 189
query7 4811 440 244 244
query8 315 166 153 153
query9 8781 3134 3134 3134
query10 478 318 292 292
query11 5977 3838 3567 3567
query12 145 89 86 86
query13 1271 450 314 314
query14 6079 3935 3658 3658
query14_1 3432 3459 3428 3428
query15 210 191 184 184
query16 937 423 438 423
query17 849 666 533 533
query18 2423 433 314 314
query19 209 170 138 138
query20 87 87 83 83
query21 213 131 109 109
query22 13047 13098 12923 12923
query23 13448 12897 12487 12487
query23_1 12610 12534 12403 12403
query24 7403 808 447 447
query24_1 469 472 473 472
query25 510 391 328 328
query26 1261 254 136 136
query27 2796 441 270 270
query28 4611 1938 1970 1938
query29 1543 577 432 432
query30 298 219 190 190
query31 1010 922 803 803
query32 137 87 91 87
query33 514 284 222 222
query34 873 840 490 490
query35 762 751 635 635
query36 804 796 749 749
query37 113 94 81 81
query38 1769 1707 1679 1679
query39 786 795 742 742
query39_1 718 729 744 729
query40 212 109 94 94
query41 76 78 78 78
query42 85 85 87 85
query43 347 357 311 311
query44 1370 688 716 688
query45 175 176 162 162
query46 828 963 566 566
query47 2874 2793 2442 2442
query48 294 305 218 218
query49 581 372 279 279
query50 697 282 211 211
query51 10790 10587 10648 10587
query52 78 78 69 69
query53 185 215 148 148
query54 243 241 173 173
query55 66 69 63 63
query56 231 215 204 204
query57 1661 1593 1555 1555
query58 279 258 266 258
query59 2087 2115 1941 1941
query60 309 230 227 227
query61 148 142 143 142
query62 382 333 275 275
query63 189 159 158 158
query64 2705 922 751 751
query65 3347 3264 3280 3264
query66 1779 418 302 302
query67 20549 20102 20267 20102
query68 3263 1013 605 605
query69 401 285 250 250
query70 837 808 747 747
query71 293 283 209 209
query72 2811 2622 2275 2275
query73 536 532 298 298
query74 4684 4500 4372 4372
query75 2352 2279 1926 1926
query76 2422 1000 593 593
query77 366 361 279 279
query78 9839 9659 9135 9135
query79 873 846 511 511
query80 476 408 325 325
query81 524 317 268 268
query82 197 135 100 100
query83 194 195 184 184
query84 286 119 106 106
query85 763 433 357 357
query86 259 237 230 230
query87 2004 1925 1799 1799
query88 3549 2655 2642 2642
query89 315 282 237 237
query90 2089 167 160 160
query91 156 150 129 129
query92 97 86 92 86
query93 991 1002 581 581
query94 453 322 288 288
query95 591 339 371 339
query96 656 499 239 239
query97 2453 2419 2322 2322
query98 157 157 144 144
query99 759 763 647 647
Total cold run time: 234465 ms
Total hot run time: 150691 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6433760836
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6545071911
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/bec29aad984da153cab0fdcc267967ab66d9c543_bec29aad984da153cab0fdcc267967ab66d9c543_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/bec29aad984da153cab0fdcc267967ab66d9c543_bec29aad984da153cab0fdcc267967ab66d9c543_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.65% (2101/2741) |
| Line Coverage | 66.00% (38792/58772) |
| Region Coverage | 53.80% (36447/67743) |
| Branch Coverage | 57.26% (11865/20722) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6594386825
<details>
<summary>TPC-H: <b>Total hot run time: 28505 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17598 5178 5043 5043
q2 2116 336 247 247
q3 10220 1408 823 823
q4 4677 471 342 342
q5 7452 805 531 531
q6 180 186 155 155
q7 750 794 600 600
q8 9258 1324 1036 1036
q9 5475 4220 4191 4191
q10 6819 1308 1043 1043
q11 430 250 227 227
q12 632 398 282 282
q13 18084 3639 3113 3113
q14 275 273 255 255
q15 q16 735 707 655 655
q17 1209 746 571 571
q18 7071 6215 6164 6164
q19 1120 971 613 613
q20 405 342 224 224
q21 4809 2129 2300 2129
q22 357 301 261 261
Total cold run time: 99672 ms
Total hot run time: 28505 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5661 5576 5568 5568
q2 697 570 541 541
q3 5232 5416 4927 4927
q4 2027 2180 1601 1601
q5 5268 5167 5606 5167
q6 270 212 159 159
q7 2340 2148 1924 1924
q8 3163 2989 2830 2830
q9 7987 8013 8060 8013
q10 3818 3720 3286 3286
q11 618 456 437 437
q12 717 728 539 539
q13 3674 4114 3281 3281
q14 324 328 302 302
q15 q16 698 719 646 646
q17 8363 7647 7285 7285
q18 12970 12162 12970 12162
q19 936 809 807 807
q20 2136 2210 1944 1944
q21 5874 4718 4830 4718
q22 502 494 402 402
Total cold run time: 73275 ms
Total hot run time: 66539 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6359396264
@xylaaaaa I pushed three commits on top of yours (after your 32552aa, which already covers the CI review bot's Arrow memory pool finding, so I left that test as you wrote it). Your fix is right for DELETE/UPDATE/MERGE today; these make the same mistake impossible to repeat silently in the next Iceberg writer:
1. `[refactor](be) Take both paths of every Iceberg file from one output location`: a new header-only `IcebergOutputLocation` holds the data location in both forms (the normalized `output_path` BE writes under, and the table's own form it commits under). The data writer, `VIcebergDeleteFileWriter` and the Puffin path all take both of their paths from it, so `_commit_path_of` is gone, and the delete file writer no longer reports its write path by default. A sink without both location fields is now rejected instead of committing a wrong path (every FE on 3.1 / 4.0 / 4.1 / 4.2 / master sets them).
2. `[improvement](fe) Reject Iceberg commit data reported outside the sink's location`: `planWrite` registers the raw location(s) each sink carries on the transaction, and `addCommitData` rejects a reported file outside them. FE then does not accept the report, so BE deletes its files and the statement fails instead of committing an unreadable path. This also means BE has to be upgraded before FE: an upgraded FE rejects the `s3://` delete file paths an unfixed BE reports.
3. `[test](regression) Check that every Iceberg write records its files under the table location`: `test_iceberg_write_file_locations` runs every write operation (INSERT, OVERWRITE PARTITION, branch INSERT, DELETE, UPDATE, MERGE, rewrite_data_files; v2 and v3) and checks every file in `$all_files` against the table's LOCATION. The REST fixture's warehouse is `s3a://`, which FE normalizes to `s3://`, so CI already ran into this bug; the existing suites just did not check the prefix.
About the red BE UT on `486c9b0`: it failed to compile `be/test/exprs/function/function_string_test.cpp`, which this PR does not touch. That was a master break, fixed by #69487 (merged a few hours ago), not this change.
The PR description is updated to cover the whole change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6409294317
<details>
<summary>ClickBench: <b>Total hot run time: 25.1 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 52c04ac75925f60e70c8415f91035b7efb97f2aa, data reload: false
query1 0.01 0.01 0.00
query2 0.15 0.08 0.08
query3 0.36 0.12 0.12
query4 1.61 0.14 0.14
query5 0.27 0.25 0.26
query6 1.16 0.65 0.65
query7 0.04 0.01 0.01
query8 0.08 0.08 0.07
query9 0.46 0.34 0.36
query10 0.57 0.56 0.62
query11 0.33 0.18 0.19
query12 0.31 0.18 0.20
query13 0.51 0.53 0.53
query14 0.93 0.92 0.92
query15 0.66 0.59 0.58
query16 0.36 0.38 0.36
query17 1.06 1.05 1.02
query18 0.29 0.29 0.28
query19 2.01 1.85 1.78
query20 0.03 0.01 0.02
query21 15.41 0.34 0.31
query22 4.94 0.13 0.13
query23 15.85 0.48 0.30
query24 2.42 0.54 0.39
query25 0.16 0.11 0.10
query26 0.77 0.27 0.21
query27 0.10 0.10 0.09
query28 3.41 0.74 0.40
query29 12.58 4.38 3.37
query30 0.35 0.16 0.15
query31 2.76 0.57 0.32
query32 3.25 0.60 0.48
query33 2.92 3.02 2.93
query34 15.92 4.03 3.43
query35 3.36 3.38 3.36
query36 0.74 0.60 0.52
query37 0.12 0.09 0.10
query38 0.07 0.06 0.06
query39 0.06 0.07 0.06
query40 0.18 0.17 0.18
query41 0.10 0.06 0.05
query42 0.06 0.06 0.06
query43 0.06 0.06 0.06
Total cold run time: 96.79 s
Total hot run time: 25.1 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub) via commits" <co...@doris.apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6546541980
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6594892602
# FE UT Coverage Report
Increment line coverage ` 100.00% (5/5)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_c8d917291673e51cea99c571dd329aab21e2bf25/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_c8d917291673e51cea99c571dd329aab21e2bf25/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6407471477
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6408564206
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_52c04ac75925f60e70c8415f91035b7efb97f2aa/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_52c04ac75925f60e70c8415f91035b7efb97f2aa/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6435394021
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/24)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6433274557
<details>
<summary>TPC-H: <b>Total hot run time: 28475 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd34e04e3c90c4ba736b6669409701880504f576, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17051 5098 5051 5051
q2 2142 311 259 259
q3 10126 1402 801 801
q4 4678 463 335 335
q5 7473 805 536 536
q6 183 181 152 152
q7 761 789 606 606
q8 9312 1328 1086 1086
q9 5571 4205 4175 4175
q10 6824 1302 1043 1043
q11 430 245 236 236
q12 657 398 278 278
q13 18121 3913 3047 3047
q14 273 280 250 250
q15 q16 729 721 671 671
q17 1204 751 596 596
q18 7083 6204 6151 6151
q19 1111 964 594 594
q20 405 338 230 230
q21 5279 2128 2273 2128
q22 350 299 250 250
Total cold run time: 99763 ms
Total hot run time: 28475 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5606 5616 5914 5616
q2 671 575 530 530
q3 5246 5409 4960 4960
q4 1992 2142 1542 1542
q5 5281 5195 5703 5195
q6 274 206 149 149
q7 2321 2134 1912 1912
q8 3133 2810 2706 2706
q9 8132 8152 8108 8108
q10 3732 3696 3260 3260
q11 612 433 429 429
q12 684 688 518 518
q13 3683 3930 3287 3287
q14 327 319 286 286
q15 q16 723 722 658 658
q17 8216 7655 6969 6969
q18 13021 12248 13016 12248
q19 901 797 798 797
q20 2173 2123 1913 1913
q21 5588 4627 4670 4627
q22 498 456 413 413
Total cold run time: 72814 ms
Total hot run time: 66123 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6547226171
<details>
<summary>ClickBench: <b>Total hot run time: 25.1 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
query1 0.01 0.01 0.01
query2 0.13 0.08 0.08
query3 0.35 0.13 0.13
query4 1.61 0.13 0.13
query5 0.27 0.24 0.25
query6 1.15 0.64 0.65
query7 0.04 0.01 0.01
query8 0.08 0.07 0.07
query9 0.46 0.35 0.37
query10 0.59 0.61 0.57
query11 0.31 0.19 0.17
query12 0.31 0.20 0.21
query13 0.52 0.51 0.54
query14 0.94 0.92 0.91
query15 0.65 0.58 0.58
query16 0.34 0.36 0.35
query17 1.00 1.01 1.00
query18 0.28 0.27 0.27
query19 1.96 1.87 1.88
query20 0.02 0.01 0.01
query21 15.39 0.34 0.30
query22 4.95 0.12 0.12
query23 15.84 0.49 0.29
query24 2.18 0.51 0.40
query25 0.16 0.11 0.10
query26 0.76 0.27 0.20
query27 0.10 0.10 0.09
query28 3.51 0.84 0.38
query29 12.62 4.32 3.36
query30 0.34 0.17 0.14
query31 2.75 0.57 0.32
query32 3.23 0.60 0.49
query33 3.06 2.99 2.94
query34 15.90 4.11 3.46
query35 3.37 3.37 3.38
query36 0.61 0.51 0.52
query37 0.11 0.10 0.09
query38 0.06 0.06 0.06
query39 0.06 0.06 0.06
query40 0.19 0.17 0.16
query41 0.11 0.06 0.05
query42 0.06 0.06 0.05
query43 0.06 0.06 0.07
Total cold run time: 96.44 s
Total hot run time: 25.1 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6594108220
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c8d917291673e51cea99c571dd329aab21e2bf25_c8d917291673e51cea99c571dd329aab21e2bf25_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c8d917291673e51cea99c571dd329aab21e2bf25_c8d917291673e51cea99c571dd329aab21e2bf25_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.65% (2101/2741) |
| Line Coverage | 66.08% (38838/58772) |
| Region Coverage | 53.87% (36495/67743) |
| Branch Coverage | 57.34% (11882/20722) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub) via commits" <co...@doris.apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6593570078
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6346990510
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/5)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_486c9b0022e81940128f84d590c8ac837fee2ebb_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_486c9b0022e81940128f84d590c8ac837fee2ebb_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6360498965
<details>
<summary>TPC-DS: <b>Total hot run time: 150969 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32552aaf9d0052f9d71e8b2d407d556112dfe8d0, data reload: false
query5 4317 573 435 435
query6 429 213 192 192
query7 4821 442 224 224
query8 329 167 150 150
query9 8764 3062 3056 3056
query10 482 311 262 262
query11 5793 4072 3590 3590
query12 149 109 87 87
query13 1275 451 322 322
query14 6083 3920 3657 3657
query14_1 3402 3423 3406 3406
query15 209 199 181 181
query16 933 451 393 393
query17 895 665 532 532
query18 2434 422 313 313
query19 199 167 138 138
query20 83 83 84 83
query21 209 130 108 108
query22 13117 12947 12906 12906
query23 13685 13144 12731 12731
query23_1 12843 12742 12447 12447
query24 7413 851 467 467
query24_1 494 468 473 468
query25 533 421 360 360
query26 1260 260 144 144
query27 2758 439 270 270
query28 4630 1917 1910 1910
query29 1575 598 456 456
query30 303 240 184 184
query31 1020 933 819 819
query32 140 96 88 88
query33 533 293 240 240
query34 915 833 491 491
query35 755 752 632 632
query36 800 789 733 733
query37 131 96 85 85
query38 1778 1703 1683 1683
query39 781 799 751 751
query39_1 733 737 738 737
query40 223 117 95 95
query41 74 69 68 68
query42 88 84 84 84
query43 342 366 316 316
query44 1293 680 685 680
query45 191 181 173 173
query46 815 1000 548 548
query47 2942 3022 2852 2852
query48 290 303 217 217
query49 577 392 288 288
query50 663 270 203 203
query51 10511 10281 10420 10281
query52 77 82 71 71
query53 181 201 153 153
query54 236 191 181 181
query55 68 65 65 65
query56 225 219 208 208
query57 1658 1610 1575 1575
query58 278 266 252 252
query59 2135 2131 1931 1931
query60 270 234 220 220
query61 145 141 141 141
query62 379 329 295 295
query63 186 155 159 155
query64 2796 919 758 758
query65 3350 3249 3278 3249
query66 1774 413 291 291
query67 20332 20127 20078 20078
query68 3041 976 598 598
query69 381 294 249 249
query70 843 794 765 765
query71 304 219 209 209
query72 2711 2597 2284 2284
query73 505 518 292 292
query74 4633 4535 4342 4342
query75 2352 2283 1905 1905
query76 2314 998 586 586
query77 328 362 261 261
query78 9763 9754 9253 9253
query79 960 887 513 513
query80 924 412 335 335
query81 585 317 282 282
query82 576 133 109 109
query83 279 194 178 178
query84 300 121 104 104
query85 1017 417 362 362
query86 352 235 229 229
query87 1992 1936 1802 1802
query88 3656 2672 2614 2614
query89 330 277 242 242
query90 1739 171 167 167
query91 167 147 129 129
query92 96 87 90 87
query93 1032 993 567 567
query94 526 317 275 275
query95 562 411 314 314
query96 669 484 231 231
query97 2443 2458 2308 2308
query98 163 154 149 149
query99 778 758 641 641
Total cold run time: 235032 ms
Total hot run time: 150969 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6412851358
<details>
<summary>ClickBench: <b>Total hot run time: 25.2 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be, data reload: false
query1 0.01 0.01 0.01
query2 0.13 0.08 0.08
query3 0.33 0.13 0.14
query4 1.61 0.14 0.14
query5 0.28 0.24 0.27
query6 1.15 0.66 0.65
query7 0.04 0.01 0.01
query8 0.09 0.07 0.07
query9 0.43 0.36 0.36
query10 0.58 0.59 0.59
query11 0.31 0.18 0.18
query12 0.32 0.19 0.18
query13 0.52 0.52 0.54
query14 0.92 0.93 0.94
query15 0.66 0.57 0.58
query16 0.35 0.36 0.37
query17 1.02 0.99 1.05
query18 0.29 0.28 0.28
query19 1.89 1.89 1.82
query20 0.06 0.01 0.01
query21 15.43 0.35 0.30
query22 4.93 0.13 0.13
query23 15.84 0.48 0.29
query24 2.18 0.57 0.38
query25 0.15 0.11 0.10
query26 0.75 0.28 0.22
query27 0.10 0.10 0.09
query28 3.53 0.77 0.39
query29 12.58 4.34 3.40
query30 0.34 0.18 0.16
query31 2.76 0.62 0.33
query32 3.23 0.60 0.47
query33 2.94 3.11 3.02
query34 15.90 4.04 3.44
query35 3.36 3.35 3.37
query36 0.58 0.49 0.47
query37 0.11 0.09 0.10
query38 0.07 0.07 0.07
query39 0.06 0.06 0.05
query40 0.19 0.17 0.17
query41 0.11 0.06 0.05
query42 0.06 0.05 0.06
query43 0.06 0.06 0.06
Total cold run time: 96.25 s
Total hot run time: 25.2 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6408825995
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/52c04ac75925f60e70c8415f91035b7efb97f2aa_52c04ac75925f60e70c8415f91035b7efb97f2aa_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/52c04ac75925f60e70c8415f91035b7efb97f2aa_52c04ac75925f60e70c8415f91035b7efb97f2aa_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.94% (2099/2728) |
| Line Coverage | 66.05% (38649/58515) |
| Region Coverage | 53.63% (36226/67549) |
| Branch Coverage | 57.26% (11806/20618) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6359760975
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6359399866
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6413706462
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/614)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6412030245
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6343683412
<details>
<summary>TPC-DS: <b>Total hot run time: 150929 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 486c9b0022e81940128f84d590c8ac837fee2ebb, data reload: false
query5 4331 579 448 448
query6 426 208 194 194
query7 4824 442 234 234
query8 320 167 152 152
query9 8770 3068 3081 3068
query10 472 324 269 269
query11 6025 3879 3568 3568
query12 145 104 86 86
query13 1272 424 337 337
query14 6103 3937 3644 3644
query14_1 3486 3405 3350 3350
query15 202 192 182 182
query16 931 432 317 317
query17 846 633 518 518
query18 2400 420 297 297
query19 191 157 132 132
query20 84 79 80 79
query21 210 127 106 106
query22 12989 12998 12843 12843
query23 13553 12678 12443 12443
query23_1 12512 12526 12456 12456
query24 7219 794 458 458
query24_1 452 461 463 461
query25 506 403 333 333
query26 1266 253 135 135
query27 2769 434 268 268
query28 4570 1927 1928 1927
query29 1529 580 445 445
query30 296 220 190 190
query31 1020 926 817 817
query32 138 109 97 97
query33 537 300 248 248
query34 917 850 492 492
query35 731 764 632 632
query36 838 814 776 776
query37 129 98 83 83
query38 1762 1731 1683 1683
query39 795 788 752 752
query39_1 736 735 745 735
query40 226 115 113 113
query41 87 69 68 68
query42 88 83 86 83
query43 341 366 310 310
query44 1286 688 687 687
query45 182 174 164 164
query46 821 958 570 570
query47 2944 2938 2800 2800
query48 307 308 223 223
query49 574 399 301 301
query50 678 287 208 208
query51 10418 10505 10526 10505
query52 83 80 71 71
query53 190 215 153 153
query54 240 206 192 192
query55 73 70 72 70
query56 263 225 236 225
query57 1637 1619 1583 1583
query58 301 283 263 263
query59 2198 2175 2002 2002
query60 301 256 235 235
query61 170 169 162 162
query62 381 329 289 289
query63 190 157 160 157
query64 2836 1021 782 782
query65 3327 3268 3293 3268
query66 1778 437 307 307
query67 20356 20274 19983 19983
query68 3246 991 608 608
query69 400 288 247 247
query70 851 791 771 771
query71 305 217 207 207
query72 2849 2557 2374 2374
query73 520 513 293 293
query74 4658 4494 4337 4337
query75 2325 2276 1922 1922
query76 2307 997 604 604
query77 358 374 278 278
query78 9878 9712 9189 9189
query79 980 839 500 500
query80 1236 445 334 334
query81 585 326 268 268
query82 597 127 114 114
query83 301 192 201 192
query84 311 127 101 101
query85 875 427 373 373
query86 405 238 229 229
query87 1992 1966 1811 1811
query88 3605 2654 2630 2630
query89 317 286 242 242
query90 1930 171 165 165
query91 155 157 127 127
query92 100 86 82 82
query93 1003 994 588 588
query94 644 300 267 267
query95 585 331 297 297
query96 660 480 232 232
query97 2444 2429 2317 2317
query98 155 157 147 147
query99 769 771 658 658
Total cold run time: 235721 ms
Total hot run time: 150929 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub) via commits" <co...@doris.apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6600711672
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6412761889
<details>
<summary>TPC-DS: <b>Total hot run time: 151253 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be, data reload: false
query5 4309 581 453 453
query6 440 209 192 192
query7 4810 443 228 228
query8 323 168 167 167
query9 8778 3140 3141 3140
query10 478 312 252 252
query11 5911 3852 3558 3558
query12 147 91 85 85
query13 1245 433 313 313
query14 6088 3963 3637 3637
query14_1 3483 3460 3422 3422
query15 203 200 205 200
query16 949 421 443 421
query17 858 668 537 537
query18 2425 438 320 320
query19 200 165 144 144
query20 89 89 85 85
query21 210 127 109 109
query22 12996 12934 12914 12914
query23 13358 13135 12481 12481
query23_1 12661 12433 12637 12433
query24 7179 848 483 483
query24_1 481 475 481 475
query25 537 429 377 377
query26 1285 267 141 141
query27 2754 433 270 270
query28 4511 1939 1940 1939
query29 1538 598 449 449
query30 290 219 190 190
query31 1015 927 814 814
query32 141 96 93 93
query33 527 315 251 251
query34 884 856 496 496
query35 732 773 639 639
query36 809 811 728 728
query37 121 101 85 85
query38 1752 1729 1658 1658
query39 783 794 742 742
query39_1 765 735 752 735
query40 220 124 98 98
query41 73 75 68 68
query42 88 87 83 83
query43 374 372 308 308
query44 1332 702 696 696
query45 185 182 171 171
query46 847 966 561 561
query47 2768 3030 2922 2922
query48 303 317 231 231
query49 615 400 286 286
query50 664 267 209 209
query51 10646 10728 10512 10512
query52 81 81 70 70
query53 189 204 154 154
query54 244 188 179 179
query55 72 69 63 63
query56 229 218 225 218
query57 1653 1634 1551 1551
query58 271 267 255 255
query59 2112 2119 1948 1948
query60 291 247 227 227
query61 150 146 145 145
query62 387 325 284 284
query63 185 159 160 159
query64 2694 966 753 753
query65 3305 3258 3270 3258
query66 1757 424 311 311
query67 20039 20381 19978 19978
query68 3026 1015 602 602
query69 385 295 278 278
query70 845 762 767 762
query71 296 243 215 215
query72 2660 2564 2282 2282
query73 518 533 297 297
query74 4621 4507 4328 4328
query75 2332 2313 1908 1908
query76 2147 1009 592 592
query77 362 376 275 275
query78 9795 9804 9190 9190
query79 980 872 513 513
query80 829 429 353 353
query81 562 321 282 282
query82 550 136 106 106
query83 280 197 188 188
query84 312 122 105 105
query85 968 424 375 375
query86 366 236 229 229
query87 2003 1931 1814 1814
query88 3647 2678 2661 2661
query89 333 287 234 234
query90 1732 169 167 167
query91 154 155 126 126
query92 100 88 92 88
query93 1044 1018 587 587
query94 542 312 269 269
query95 561 391 314 314
query96 668 486 233 233
query97 2426 2443 2323 2323
query98 165 153 146 146
query99 757 769 651 651
Total cold run time: 233550 ms
Total hot run time: 151253 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6547170060
<details>
<summary>TPC-DS: <b>Total hot run time: 151011 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
query5 4311 576 453 453
query6 419 202 186 186
query7 4800 431 222 222
query8 323 186 147 147
query9 8764 3221 3198 3198
query10 476 307 252 252
query11 6023 3844 3559 3559
query12 157 92 86 86
query13 1263 440 324 324
query14 6085 3916 3672 3672
query14_1 3395 3367 3399 3367
query15 201 198 178 178
query16 936 433 410 410
query17 839 644 515 515
query18 2410 404 296 296
query19 185 160 124 124
query20 81 83 79 79
query21 206 125 102 102
query22 12906 12983 12774 12774
query23 13335 12851 12412 12412
query23_1 12534 12429 12445 12429
query24 7176 840 467 467
query24_1 500 469 478 469
query25 535 428 367 367
query26 1249 261 140 140
query27 2759 436 272 272
query28 4560 2024 2003 2003
query29 1537 617 450 450
query30 298 210 182 182
query31 1014 936 820 820
query32 141 95 93 93
query33 530 313 267 267
query34 912 856 508 508
query35 723 763 641 641
query36 881 803 726 726
query37 126 98 90 90
query38 1782 1770 1656 1656
query39 806 794 745 745
query39_1 732 728 743 728
query40 222 121 99 99
query41 72 69 69 69
query42 94 84 82 82
query43 378 370 316 316
query44 1329 708 709 708
query45 174 183 170 170
query46 822 959 557 557
query47 2881 2941 2672 2672
query48 319 305 221 221
query49 588 403 304 304
query50 701 275 226 226
query51 10525 10472 10587 10472
query52 82 80 80 80
query53 187 208 158 158
query54 268 205 184 184
query55 70 67 66 66
query56 256 224 237 224
query57 1645 1619 1549 1549
query58 297 269 282 269
query59 2147 2216 1974 1974
query60 309 265 229 229
query61 177 161 171 161
query62 395 337 296 296
query63 191 164 160 160
query64 2843 940 756 756
query65 3344 3257 3247 3247
query66 1761 408 313 313
query67 20279 20044 20090 20044
query68 2988 1018 607 607
query69 394 279 256 256
query70 849 767 772 767
query71 302 225 203 203
query72 2683 2567 2332 2332
query73 513 540 289 289
query74 4644 4529 4328 4328
query75 2314 2275 1924 1924
query76 2186 1011 636 636
query77 349 360 269 269
query78 9955 9755 9157 9157
query79 999 858 524 524
query80 1246 406 346 346
query81 601 318 278 278
query82 566 133 109 109
query83 310 196 178 178
query84 313 118 100 100
query85 891 446 371 371
query86 413 234 234 234
query87 2032 1916 1818 1818
query88 3849 2795 2782 2782
query89 333 286 243 243
query90 1937 164 166 164
query91 158 145 123 123
query92 95 86 89 86
query93 1010 991 571 571
query94 636 302 273 273
query95 569 332 360 332
query96 712 501 244 244
query97 2459 2451 2313 2313
query98 177 157 145 145
query99 764 759 642 642
Total cold run time: 235274 ms
Total hot run time: 151011 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6312114398
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "github-actions[bot] (via GitHub) via commits" <co...@doris.apache.org>.
github-actions[bot] commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6601749373
Codex automated review failed and did not complete.
Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41616249670
Please inspect the workflow logs and rerun the review after the underlying issue is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6336011861
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6356693907
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6410371273
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6594516287
<details>
<summary>TPC-DS: <b>Total hot run time: 152067 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
query5 4323 608 461 461
query6 438 203 190 190
query7 4799 449 231 231
query8 323 174 182 174
query9 8739 3232 3223 3223
query10 483 303 260 260
query11 5806 3902 3608 3608
query12 147 96 88 88
query13 1245 456 325 325
query14 6141 3972 3667 3667
query14_1 3465 3428 3530 3428
query15 214 204 182 182
query16 945 424 410 410
query17 897 651 543 543
query18 2421 425 326 326
query19 208 177 141 141
query20 86 97 85 85
query21 214 131 110 110
query22 12994 13074 12760 12760
query23 13588 12961 12428 12428
query23_1 12591 12611 12619 12611
query24 7378 827 481 481
query24_1 492 498 491 491
query25 555 432 376 376
query26 1267 256 144 144
query27 2774 448 282 282
query28 4607 2061 2053 2053
query29 1564 601 466 466
query30 320 228 185 185
query31 1019 939 836 836
query32 157 96 97 96
query33 522 311 250 250
query34 917 858 516 516
query35 746 770 649 649
query36 825 808 738 738
query37 126 99 86 86
query38 1793 1728 1684 1684
query39 797 799 739 739
query39_1 742 726 751 726
query40 233 121 99 99
query41 72 71 68 68
query42 89 80 81 80
query43 350 361 317 317
query44 1342 717 717 717
query45 188 182 167 167
query46 851 978 567 567
query47 3032 2981 2660 2660
query48 313 322 249 249
query49 565 427 289 289
query50 679 288 209 209
query51 10490 10513 10791 10513
query52 80 76 66 66
query53 182 215 148 148
query54 253 195 189 189
query55 70 67 66 66
query56 236 225 213 213
query57 1675 1646 1579 1579
query58 285 262 251 251
query59 2113 2233 1987 1987
query60 293 239 220 220
query61 149 150 146 146
query62 381 340 282 282
query63 190 157 157 157
query64 2823 949 813 813
query65 3359 3273 3305 3273
query66 1788 413 304 304
query67 20168 20385 20132 20132
query68 3280 990 652 652
query69 400 284 251 251
query70 860 796 793 793
query71 291 250 209 209
query72 2830 2621 2352 2352
query73 528 529 301 301
query74 4646 4508 4359 4359
query75 2390 2298 1906 1906
query76 2392 1012 597 597
query77 358 357 269 269
query78 9970 9847 9200 9200
query79 986 860 547 547
query80 818 472 344 344
query81 588 314 267 267
query82 580 131 104 104
query83 306 207 188 188
query84 312 129 100 100
query85 906 426 395 395
query86 349 240 229 229
query87 1991 1934 1829 1829
query88 3840 2813 2799 2799
query89 337 282 246 246
query90 1807 170 160 160
query91 165 153 131 131
query92 98 90 91 90
query93 1009 1020 592 592
query94 548 314 265 265
query95 582 407 303 303
query96 696 497 238 238
query97 2449 2424 2302 2302
query98 162 152 166 152
query99 787 763 650 650
Total cold run time: 236007 ms
Total hot run time: 152067 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6435390891
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (23/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_cd34e04e3c90c4ba736b6669409701880504f576_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 77.50% (36123/46611) |
| Line Coverage | 62.77% (408574/650927) |
| Region Coverage | 59.22% (345440/583345) |
| Branch Coverage | 60.11% (159754/265786) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6433454897
<details>
<summary>ClickBench: <b>Total hot run time: 24.41 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd34e04e3c90c4ba736b6669409701880504f576, data reload: false
query1 0.00 0.00 0.01
query2 0.09 0.05 0.05
query3 0.26 0.09 0.09
query4 1.62 0.09 0.10
query5 0.23 0.25 0.24
query6 1.16 0.69 0.68
query7 0.05 0.01 0.01
query8 0.06 0.04 0.04
query9 0.41 0.35 0.36
query10 0.57 0.61 0.58
query11 0.20 0.15 0.14
query12 0.18 0.15 0.15
query13 0.49 0.49 0.48
query14 1.03 1.01 1.00
query15 0.64 0.61 0.61
query16 0.31 0.36 0.34
query17 1.13 1.12 1.13
query18 0.21 0.19 0.18
query19 2.10 1.99 2.02
query20 0.02 0.02 0.01
query21 15.50 0.19 0.14
query22 4.87 0.06 0.06
query23 16.28 0.33 0.12
query24 2.89 0.63 0.38
query25 0.10 0.06 0.04
query26 1.24 0.23 0.15
query27 0.06 0.03 0.04
query28 3.05 0.75 0.36
query29 12.51 4.09 3.27
query30 0.28 0.14 0.10
query31 2.77 0.56 0.32
query32 3.23 0.60 0.51
query33 3.18 3.20 3.21
query34 15.35 3.97 3.39
query35 3.34 3.36 3.34
query36 0.56 0.46 0.44
query37 0.10 0.10 0.10
query38 0.06 0.05 0.04
query39 0.04 0.03 0.03
query40 0.20 0.15 0.15
query41 0.09 0.03 0.03
query42 0.03 0.04 0.03
query43 0.04 0.03 0.04
Total cold run time: 96.53 s
Total hot run time: 24.41 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub) via commits" <co...@doris.apache.org>.
morningman merged PR #69392:
URL: https://github.com/apache/doris/pull/69392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub) via commits" <co...@doris.apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6544695819
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6355562830
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6360581483
<details>
<summary>ClickBench: <b>Total hot run time: 24.47 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32552aaf9d0052f9d71e8b2d407d556112dfe8d0, data reload: false
query1 0.00 0.00 0.01
query2 0.10 0.05 0.05
query3 0.26 0.10 0.09
query4 1.61 0.10 0.10
query5 0.23 0.25 0.25
query6 1.17 0.69 0.68
query7 0.04 0.01 0.01
query8 0.05 0.03 0.04
query9 0.40 0.34 0.37
query10 0.58 0.60 0.58
query11 0.20 0.14 0.13
query12 0.19 0.15 0.14
query13 0.48 0.50 0.49
query14 1.02 1.03 1.01
query15 0.63 0.61 0.61
query16 0.34 0.35 0.34
query17 1.14 1.14 1.11
query18 0.22 0.18 0.20
query19 2.07 2.00 1.97
query20 0.02 0.01 0.02
query21 15.50 0.19 0.14
query22 4.96 0.06 0.05
query23 16.28 0.32 0.13
query24 2.95 0.70 0.65
query25 0.11 0.06 0.06
query26 1.65 0.22 0.15
query27 0.05 0.05 0.05
query28 3.83 0.71 0.38
query29 12.49 4.05 3.23
query30 0.28 0.14 0.10
query31 2.78 0.55 0.33
query32 3.23 0.60 0.51
query33 3.14 3.13 3.25
query34 15.28 3.97 3.38
query35 3.36 3.32 3.30
query36 0.56 0.42 0.43
query37 0.09 0.06 0.07
query38 0.05 0.03 0.03
query39 0.04 0.03 0.03
query40 0.18 0.15 0.14
query41 0.08 0.04 0.02
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.72 s
Total hot run time: 24.47 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69392:
URL: https://github.com/apache/doris/pull/69392#discussion_r4357250839
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorTransaction.java:
##########
@@ -581,10 +585,46 @@ public void addCommitData(byte[] commitFragment) {
throw new DorisConnectorException("failed to deserialize Iceberg commit data", e);
}
synchronized (this) {
+ checkCommitLocation(data);
Review Comment:
[P2] Fail the FE statement when rejecting a commit location
When this guard rejects a final report (for example, a pre-fix BE reports an `s3://` delete file for an `oss://` table), throwing here does not fail the waiting FE statement. `QeProcessorImpl.reportExecStatus` catches the exception and returns a negative ACK without setting the coordinator error; both coordinator implementations leave that fragment incomplete and its completion latch unchanged. BE cleans the files and calls `QueryContext::cancel`, but the final report was scheduled after all fragment tasks closed, so with no live sibling there is no subsequent error report to wake FE. The statement waits until its execution timeout and loses the location error. Propagate this terminal validation failure into the FE query status and release its waiter before returning the rejected ACK; cover this with a final-report rejection test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6316804546
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6432436812
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6413514535
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (23/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 77.43% (36074/46587) |
| Line Coverage | 62.62% (407405/650625) |
| Region Coverage | 58.97% (343589/582649) |
| Branch Coverage | 60.00% (159280/265470) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6545921796
# BE UT Coverage Report
Increment line coverage `100.00% (23/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/bec29aad984da153cab0fdcc267967ab66d9c543_bec29aad984da153cab0fdcc267967ab66d9c543/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/bec29aad984da153cab0fdcc267967ab66d9c543_bec29aad984da153cab0fdcc267967ab66d9c543/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 67.47% (32908/48772) |
| Line Coverage | 52.42% (349526/666778) |
| Region Coverage | 47.77% (283793/594111) |
| Branch Coverage | 49.46% (134119/271146) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub) via commits" <co...@doris.apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6540977883
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6406956651
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "shuke987 (via GitHub)" <gi...@apache.org>.
shuke987 commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6408371131
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6412575340
<details>
<summary>TPC-H: <b>Total hot run time: 28266 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc89f8497ebb3b28c1caab2b8ed9b1e49faa06be, data reload: false
------ Round 1 ----------------------------------
============================================
q1 16721 5129 5029 5029
q2 2118 301 246 246
q3 10057 1380 824 824
q4 4666 461 338 338
q5 7490 797 521 521
q6 181 184 148 148
q7 732 787 590 590
q8 9334 1318 1039 1039
q9 5478 4158 4199 4158
q10 6833 1287 1028 1028
q11 435 241 230 230
q12 635 398 277 277
q13 18140 3596 3021 3021
q14 274 279 250 250
q15 q16 725 714 668 668
q17 1177 731 581 581
q18 7015 6161 6141 6141
q19 1096 956 600 600
q20 375 337 227 227
q21 5190 2096 2249 2096
q22 343 297 254 254
Total cold run time: 99015 ms
Total hot run time: 28266 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5564 5571 5580 5571
q2 683 569 515 515
q3 5184 5362 4987 4987
q4 1951 2129 1512 1512
q5 5328 5157 5362 5157
q6 254 200 166 166
q7 2410 2050 1875 1875
q8 2977 2693 2697 2693
q9 7922 8075 8010 8010
q10 3687 3721 3240 3240
q11 601 433 570 433
q12 742 695 514 514
q13 3472 3823 3238 3238
q14 331 309 299 299
q15 q16 692 727 633 633
q17 8290 7349 6858 6858
q18 13005 12172 12879 12172
q19 875 770 760 760
q20 2138 2142 1927 1927
q21 5533 4573 4650 4573
q22 498 442 433 433
Total cold run time: 72137 ms
Total hot run time: 65566 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6409110819
<details>
<summary>TPC-H: <b>Total hot run time: 28555 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 52c04ac75925f60e70c8415f91035b7efb97f2aa, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17647 5148 5053 5053
q2 2131 362 250 250
q3 10175 1391 836 836
q4 4672 468 332 332
q5 7438 805 536 536
q6 175 179 155 155
q7 779 800 595 595
q8 9256 1342 1099 1099
q9 5503 4191 4195 4191
q10 6848 1302 1031 1031
q11 431 244 228 228
q12 636 395 290 290
q13 18070 3616 3056 3056
q14 268 280 237 237
q15 q16 723 720 639 639
q17 1220 744 599 599
q18 7140 6200 6129 6129
q19 1112 982 622 622
q20 391 336 225 225
q21 5386 2195 2305 2195
q22 357 310 257 257
Total cold run time: 100358 ms
Total hot run time: 28555 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5631 5634 5637 5634
q2 695 578 507 507
q3 5203 5357 4941 4941
q4 2033 2150 1545 1545
q5 5254 5169 5762 5169
q6 257 197 151 151
q7 2312 2127 1889 1889
q8 3062 2886 2737 2737
q9 7973 8044 8006 8006
q10 3729 3699 3241 3241
q11 606 440 426 426
q12 687 690 519 519
q13 3470 4015 3207 3207
q14 330 309 312 309
q15 q16 703 710 638 638
q17 8232 7436 7379 7379
q18 12957 12236 12984 12236
q19 907 797 769 769
q20 2151 2199 1910 1910
q21 5736 4620 4730 4620
q22 492 451 421 421
Total cold run time: 72420 ms
Total hot run time: 66254 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "xylaaaaa (via GitHub)" <gi...@apache.org>.
xylaaaaa commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6410277333
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6433393359
<details>
<summary>TPC-DS: <b>Total hot run time: 150857 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd34e04e3c90c4ba736b6669409701880504f576, data reload: false
query5 4311 592 462 462
query6 422 207 190 190
query7 4810 439 223 223
query8 312 163 143 143
query9 8773 3067 3077 3067
query10 466 302 266 266
query11 6042 3833 3548 3548
query12 155 93 90 90
query13 1252 446 309 309
query14 6091 3878 3606 3606
query14_1 3385 3327 3372 3327
query15 201 197 181 181
query16 970 421 391 391
query17 837 639 509 509
query18 2405 416 292 292
query19 192 169 156 156
query20 83 91 82 82
query21 203 125 101 101
query22 12909 13047 12694 12694
query23 13435 12819 12458 12458
query23_1 12579 12537 12658 12537
query24 7291 814 469 469
query24_1 486 482 474 474
query25 540 426 367 367
query26 1256 255 141 141
query27 2774 437 270 270
query28 4623 1914 1907 1907
query29 1554 610 452 452
query30 279 217 181 181
query31 1002 923 819 819
query32 142 93 91 91
query33 520 292 237 237
query34 903 851 498 498
query35 711 739 642 642
query36 818 813 703 703
query37 127 95 75 75
query38 1765 1774 1649 1649
query39 797 796 744 744
query39_1 742 729 749 729
query40 224 114 96 96
query41 68 67 63 63
query42 89 84 84 84
query43 342 369 339 339
query44 1282 685 685 685
query45 176 187 163 163
query46 805 974 557 557
query47 2869 2866 2834 2834
query48 303 301 223 223
query49 597 391 299 299
query50 658 279 212 212
query51 10451 10494 10590 10494
query52 78 83 71 71
query53 185 212 150 150
query54 241 213 176 176
query55 74 70 68 68
query56 235 219 227 219
query57 1650 1612 1551 1551
query58 282 278 269 269
query59 2179 2227 2049 2049
query60 311 257 236 236
query61 172 170 160 160
query62 399 334 292 292
query63 192 160 161 160
query64 2851 1052 882 882
query65 3324 3282 3275 3275
query66 1786 438 317 317
query67 20293 20105 19962 19962
query68 3357 1004 609 609
query69 409 301 256 256
query70 870 843 761 761
query71 306 243 226 226
query72 3041 2713 2358 2358
query73 537 517 297 297
query74 4666 4492 4323 4323
query75 2331 2298 1899 1899
query76 2367 1004 596 596
query77 345 391 251 251
query78 9858 9689 9189 9189
query79 977 859 510 510
query80 1224 440 348 348
query81 591 317 275 275
query82 573 167 105 105
query83 309 195 193 193
query84 309 128 102 102
query85 867 427 359 359
query86 404 241 232 232
query87 1985 1926 1804 1804
query88 3632 2649 2645 2645
query89 329 285 244 244
query90 1935 175 171 171
query91 167 153 129 129
query92 99 93 89 89
query93 991 997 586 586
query94 633 296 278 278
query95 557 338 300 300
query96 661 481 229 229
query97 2481 2418 2362 2362
query98 164 158 148 148
query99 767 763 652 652
Total cold run time: 235569 ms
Total hot run time: 150857 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6360334930
<details>
<summary>TPC-H: <b>Total hot run time: 28398 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32552aaf9d0052f9d71e8b2d407d556112dfe8d0, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17585 5184 5053 5053
q2 2109 330 280 280
q3 10220 1371 819 819
q4 4684 460 336 336
q5 7441 783 529 529
q6 183 182 151 151
q7 748 798 610 610
q8 9270 1303 1049 1049
q9 5445 4174 4166 4166
q10 6820 1288 1028 1028
q11 443 246 239 239
q12 636 393 285 285
q13 18079 3680 3026 3026
q14 280 273 255 255
q15 q16 717 719 673 673
q17 1190 729 579 579
q18 7086 6210 6121 6121
q19 1112 952 573 573
q20 370 338 229 229
q21 5181 2151 2245 2151
q22 355 298 246 246
Total cold run time: 99954 ms
Total hot run time: 28398 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5599 5638 5623 5623
q2 681 576 535 535
q3 5246 5342 4913 4913
q4 2018 2145 1551 1551
q5 5277 5140 5601 5140
q6 253 215 167 167
q7 2360 2091 1900 1900
q8 3063 2941 2742 2742
q9 8031 7963 8042 7963
q10 3720 3657 3180 3180
q11 618 438 408 408
q12 688 693 529 529
q13 3519 3823 3372 3372
q14 336 319 290 290
q15 q16 704 706 671 671
q17 8155 7401 7336 7336
q18 13017 12256 12974 12256
q19 936 829 787 787
q20 2123 2139 1943 1943
q21 5536 4608 4673 4608
q22 485 442 402 402
Total cold run time: 72365 ms
Total hot run time: 66316 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6343472404
<details>
<summary>TPC-H: <b>Total hot run time: 28713 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 486c9b0022e81940128f84d590c8ac837fee2ebb, data reload: false
------ Round 1 ----------------------------------
============================================
q1 16688 5146 5068 5068
q2 2102 310 260 260
q3 9972 1384 823 823
q4 4666 464 330 330
q5 7498 791 534 534
q6 184 179 153 153
q7 748 793 605 605
q8 9327 1347 1019 1019
q9 5518 4210 4166 4166
q10 6845 1303 1017 1017
q11 427 250 234 234
q12 646 401 294 294
q13 18103 3670 2995 2995
q14 277 272 246 246
q15 q16 750 706 673 673
q17 1220 748 578 578
q18 7032 6215 6139 6139
q19 1121 949 605 605
q20 399 345 227 227
q21 5563 2481 2459 2459
q22 370 335 288 288
Total cold run time: 99456 ms
Total hot run time: 28713 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5960 5891 6091 5891
q2 708 609 588 588
q3 5622 5737 5361 5361
q4 2032 2145 1555 1555
q5 5840 5929 5550 5550
q6 237 187 140 140
q7 2239 1999 1737 1737
q8 3051 2692 2627 2627
q9 7898 7906 7611 7611
q10 3608 3562 3160 3160
q11 578 408 396 396
q12 661 689 500 500
q13 3222 3588 3012 3012
q14 279 298 267 267
q15 q16 670 683 618 618
q17 7715 7020 6898 6898
q18 12944 12136 12950 12136
q19 915 783 793 783
q20 2146 2193 1929 1929
q21 5584 4595 4661 4595
q22 494 475 404 404
Total cold run time: 72403 ms
Total hot run time: 65758 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6545364294
<details>
<summary>TPC-DS: <b>Total hot run time: 151287 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bec29aad984da153cab0fdcc267967ab66d9c543, data reload: false
query5 4310 587 451 451
query6 423 206 193 193
query7 4804 444 231 231
query8 349 172 148 148
query9 8773 3198 3206 3198
query10 491 310 271 271
query11 5985 3879 3567 3567
query12 154 90 87 87
query13 1251 460 351 351
query14 6099 3952 3642 3642
query14_1 3437 3383 3460 3383
query15 205 202 182 182
query16 938 453 430 430
query17 863 657 532 532
query18 2434 423 312 312
query19 203 170 134 134
query20 87 85 84 84
query21 212 130 108 108
query22 13033 12940 13113 12940
query23 13347 12774 12327 12327
query23_1 12544 12492 12560 12492
query24 7281 835 476 476
query24_1 499 499 497 497
query25 537 449 368 368
query26 1280 262 144 144
query27 2762 456 273 273
query28 4553 2019 2017 2017
query29 1587 601 457 457
query30 307 224 189 189
query31 1020 932 825 825
query32 155 97 97 97
query33 537 307 249 249
query34 891 876 517 517
query35 743 753 640 640
query36 871 813 720 720
query37 127 97 86 86
query38 1783 1751 1670 1670
query39 806 799 753 753
query39_1 730 738 746 738
query40 234 122 97 97
query41 93 70 69 69
query42 94 84 82 82
query43 349 359 309 309
query44 1313 689 702 689
query45 178 177 169 169
query46 822 1000 559 559
query47 2916 2593 2841 2593
query48 290 305 219 219
query49 574 388 312 312
query50 678 272 212 212
query51 10816 10613 10740 10613
query52 76 89 67 67
query53 184 208 154 154
query54 226 210 183 183
query55 80 65 67 65
query56 246 224 205 205
query57 1666 1581 1580 1580
query58 280 269 253 253
query59 2213 2198 1969 1969
query60 296 246 224 224
query61 155 145 146 145
query62 386 330 285 285
query63 188 154 158 154
query64 2708 968 776 776
query65 3324 3243 3259 3243
query66 1765 421 305 305
query67 20090 20051 20009 20009
query68 2982 1000 610 610
query69 382 294 245 245
query70 864 770 769 769
query71 305 230 230 230
query72 2700 2604 2296 2296
query73 509 521 297 297
query74 4596 4488 4343 4343
query75 2318 2318 1909 1909
query76 2242 1011 580 580
query77 338 362 276 276
query78 9730 9687 9120 9120
query79 990 839 540 540
query80 829 438 357 357
query81 564 320 271 271
query82 551 135 109 109
query83 299 200 177 177
query84 307 121 98 98
query85 907 427 370 370
query86 359 238 231 231
query87 1995 1957 1842 1842
query88 3745 2791 2761 2761
query89 327 283 255 255
query90 1848 174 168 168
query91 166 152 129 129
query92 104 92 82 82
query93 994 996 576 576
query94 515 315 268 268
query95 571 324 321 321
query96 687 496 240 240
query97 2484 2444 2347 2347
query98 160 152 147 147
query99 762 768 650 650
Total cold run time: 234502 ms
Total hot run time: 151287 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "morningman (via GitHub) via commits" <co...@doris.apache.org>.
morningman commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6619300037
<!-- doris-repo-review:v1:begin -->
### Local pipeline review — ✅ PASS
```yaml
schema: doris-repo-review/v1
status: PASS
pr: apache/doris#69392
commit: c8d917291673e51cea99c571dd329aab21e2bf25
base: 8e10b235cf030d486b6840cfd64b6218ab515e6d
reviewed_at: 2026-11-04T08:40+08:00
reviewer: morningman
model: claude-opus-5-5
effort: max
findings: {blocker: 0, major: 0, minor: 2, nit: 5}
rounds: 1
converged: true
```
**Notes for maintainers**
- F-01 (wording, undeclared behaviour change) `fe/fe-core/src/main/java/org/apache/doris/qe/QeProcessorImpl.java:361-365`: since c8d91729167, FE cancels the coordinator whenever it rejects a final external-file report. This applies to Hive, Iceberg, MaxCompute and Paimon alike. A statement that used to hang until `insert_timeout` now fails at once and carries the reason. The change is correct, and no statement that would have succeeded can fail because of it. But the PR body does not declare it, and its line 47 still credits BE's cancel. Worth fixing before the squash merge.
- Backport (`dev/4.1.x`, `dev/4.2.x`): only the BE half picks as is. branch-4.1/4.2 plan Iceberg writes through fe-core `IcebergTransaction` and the legacy `planner/Iceberg*Sink`. Neither branch has the fe-connector-iceberg write path or the `rejectedExternalFileReport` ack handling in `QeProcessorImpl`. So the FE commit-location gate (`fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorTransaction.java:660-671`) and the cancel have to be re-done there.
- Not verified by this review: nothing was built or run. CI at this head passed every new BE/FE case and `test_iceberg_write_file_locations`. The red `check_coverage_fe` is the exit-4 infra failure seen across PRs. No real OSS or Azure storage was exercised.
<sub>Reviewed locally with the `doris-repo-review` pipeline. Repository policy may accept this receipt for the matching commit; it is not a human Apache approval.</sub>
<!-- doris-repo-review:v1:end -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6317626524
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0_129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0_129e0ee8b275ad7a3b5d4f5238c6ec62e19968b0/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 66.45% (31855/47939) |
| Line Coverage | 51.23% (335593/655129) |
| Region Coverage | 46.60% (271341/582282) |
| Branch Coverage | 48.33% (127986/264796) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6312112151
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69392:
URL: https://github.com/apache/doris/pull/69392#discussion_r4355478721
##########
be/test/exec/sink/viceberg_delete_sink_test.cpp:
##########
@@ -533,36 +564,120 @@ TEST_F(VIcebergDeleteSinkTest, TestValidateDeletionVectorContentSize) {
ASSERT_TRUE(unsupported_status.is<ErrorCode::NOT_IMPLEMENTED_ERROR>()) << unsupported_status;
}
+TEST_F(VIcebergDeleteSinkTest, TestWritePositionDeletesCommitsTableLocation) {
+ std::filesystem::path temp_dir = std::filesystem::temp_directory_path() /
+ ("iceberg_delete_sink_test_" + generate_uuid_string());
+ ASSERT_TRUE(std::filesystem::create_directories(temp_dir));
+
+ // Write locally, but record the table's original URI in the commit metadata.
+ const std::string table_location = "oss://bucket/warehouse/table/data/";
+ const std::string data_file_path = table_location + "file1.parquet";
+ TDataSink t_data_sink = build_local_delete_sink(temp_dir.string(), 2);
+ t_data_sink.iceberg_delete_sink.__set_table_location(table_location);
+ RuntimeState state;
+ VExprContextSPtrs output_exprs;
+ auto sink = std::make_shared<VIcebergDeleteSink>(t_data_sink, output_exprs, nullptr, nullptr);
+ ObjectPool pool;
+ ASSERT_TRUE(sink->init_properties(&pool).ok());
+ sink->_state = &state;
+
+ std::map<std::string, IcebergFileDeletion> file_deletions;
+ auto [it, inserted] =
+ file_deletions.emplace(data_file_path, IcebergFileDeletion(1, "[\"p=1\"]"));
+ ASSERT_TRUE(inserted);
+ it->second.rows_to_delete.add(uint32_t {10});
+ it->second.rows_to_delete.add(uint32_t {20});
+
+ const auto status = sink->_write_position_delete_files(file_deletions);
Review Comment:
[P2] Initialize the Arrow memory pool before the Parquet write
This new test reaches the real Parquet writer, but its fixture never initializes `ExecEnv::_arrow_memory_pool`: it defaults to null, and `run_all_tests.cpp` only calls `init_mem_tracker()`, which does not set it. `VIcebergDeleteFileWriter` enables dictionary encoding, and `VParquetWriter::_parse_properties()` explicitly forwards that null pool into `WriterProperties`. In Arrow 24, the [dictionary encoder](https://github.com/apache/arrow/blob/apache-arrow-24.0.0/cpp/src/parquet/encoder.cc#L428-L432) retains it in `buffered_indices_`; the first value's insertion reaches the [STL allocator](https://github.com/apache/arrow/blob/apache-arrow-24.0.0/cpp/src/arrow/stl_allocator.h#L49-L61), which calls `pool_->Allocate` without a null fallback. Thus running this test in isolation crashes before it can verify the commit-path assertions. Save, initialize, and restore the pool as `VParquetWriterTest::SetUp/TearDown` already does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6343294408
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/486c9b0022e81940128f84d590c8ac837fee2ebb_486c9b0022e81940128f84d590c8ac837fee2ebb_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/486c9b0022e81940128f84d590c8ac837fee2ebb_486c9b0022e81940128f84d590c8ac837fee2ebb_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 78.02% (2080/2666) |
| Line Coverage | 66.24% (38335/57876) |
| Region Coverage | 53.65% (35976/67052) |
| Branch Coverage | 57.22% (11600/20274) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6344829224
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_486c9b0022e81940128f84d590c8ac837fee2ebb/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_486c9b0022e81940128f84d590c8ac837fee2ebb/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Commit Iceberg delete files under the table's own location [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6343789983
<details>
<summary>ClickBench: <b>Total hot run time: 24.26 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 486c9b0022e81940128f84d590c8ac837fee2ebb, data reload: false
query1 0.00 0.01 0.00
query2 0.09 0.06 0.05
query3 0.26 0.09 0.10
query4 1.61 0.10 0.11
query5 0.23 0.24 0.24
query6 1.17 0.68 0.68
query7 0.03 0.01 0.00
query8 0.05 0.04 0.04
query9 0.40 0.36 0.34
query10 0.58 0.58 0.60
query11 0.20 0.14 0.14
query12 0.18 0.14 0.14
query13 0.48 0.49 0.49
query14 1.02 1.02 1.01
query15 0.63 0.61 0.60
query16 0.35 0.36 0.34
query17 1.14 1.14 1.12
query18 0.21 0.19 0.18
query19 2.00 2.04 1.92
query20 0.02 0.02 0.01
query21 15.49 0.20 0.13
query22 4.81 0.05 0.06
query23 16.26 0.33 0.12
query24 3.10 0.49 0.32
query25 0.11 0.06 0.04
query26 0.81 0.21 0.14
query27 0.06 0.04 0.05
query28 3.63 0.80 0.36
query29 12.48 4.14 3.29
query30 0.28 0.13 0.12
query31 2.77 0.55 0.32
query32 3.22 0.58 0.49
query33 3.14 3.26 3.17
query34 15.33 3.97 3.43
query35 3.33 3.34 3.34
query36 0.56 0.45 0.42
query37 0.12 0.09 0.11
query38 0.05 0.04 0.04
query39 0.05 0.04 0.03
query40 0.18 0.16 0.15
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.05 0.04 0.04
Total cold run time: 96.6 s
Total hot run time: 24.26 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6594580698
<details>
<summary>ClickBench: <b>Total hot run time: 25.29 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
query1 0.00 0.00 0.01
query2 0.14 0.09 0.09
query3 0.35 0.13 0.12
query4 1.60 0.13 0.14
query5 0.27 0.25 0.24
query6 1.16 0.64 0.64
query7 0.04 0.01 0.00
query8 0.09 0.07 0.07
query9 0.45 0.36 0.36
query10 0.59 0.57 0.62
query11 0.32 0.18 0.19
query12 0.31 0.19 0.19
query13 0.54 0.53 0.52
query14 0.95 0.93 0.93
query15 0.68 0.58 0.58
query16 0.36 0.36 0.36
query17 0.97 1.02 1.03
query18 0.29 0.28 0.27
query19 1.92 1.84 1.88
query20 0.02 0.02 0.02
query21 15.41 0.34 0.30
query22 4.91 0.13 0.13
query23 15.89 0.48 0.30
query24 2.40 0.52 0.41
query25 0.15 0.10 0.10
query26 0.77 0.26 0.21
query27 0.10 0.10 0.09
query28 3.40 0.83 0.39
query29 12.60 4.44 3.44
query30 0.34 0.17 0.13
query31 2.76 0.62 0.33
query32 3.23 0.62 0.49
query33 2.95 3.00 3.05
query34 15.86 4.12 3.46
query35 3.38 3.39 3.40
query36 0.58 0.48 0.49
query37 0.12 0.09 0.10
query38 0.07 0.06 0.07
query39 0.06 0.06 0.05
query40 0.20 0.17 0.16
query41 0.12 0.05 0.05
query42 0.07 0.05 0.05
query43 0.06 0.06 0.05
Total cold run time: 96.48 s
Total hot run time: 25.29 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6598886113
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (23/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69392_c8d917291673e51cea99c571dd329aab21e2bf25_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69392_c8d917291673e51cea99c571dd329aab21e2bf25_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 77.70% (35878/46176) |
| Line Coverage | 62.94% (402859/640113) |
| Region Coverage | 59.51% (340960/572963) |
| Branch Coverage | 60.59% (157742/260325) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6545250257
<details>
<summary>TPC-H: <b>Total hot run time: 28642 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bec29aad984da153cab0fdcc267967ab66d9c543, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17411 5076 5017 5017
q2 2115 317 256 256
q3 10061 1388 825 825
q4 4672 463 335 335
q5 7463 786 528 528
q6 182 185 155 155
q7 753 799 598 598
q8 9275 1313 1067 1067
q9 5468 4180 4183 4180
q10 6816 1292 1007 1007
q11 435 244 232 232
q12 658 399 283 283
q13 18067 3654 3041 3041
q14 279 273 248 248
q15 q16 718 723 653 653
q17 1172 735 571 571
q18 6951 6148 6079 6079
q19 1098 954 611 611
q20 377 326 226 226
q21 5494 2498 2469 2469
q22 385 353 261 261
Total cold run time: 99850 ms
Total hot run time: 28642 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5831 6025 5997 5997
q2 712 571 567 567
q3 5598 5627 5397 5397
q4 2057 2153 1544 1544
q5 5704 5715 5764 5715
q6 240 190 147 147
q7 2243 2031 1785 1785
q8 2962 2456 2502 2456
q9 7452 7463 7454 7454
q10 3614 3557 3121 3121
q11 590 432 381 381
q12 669 661 506 506
q13 3334 3538 3036 3036
q14 280 289 256 256
q15 q16 659 671 615 615
q17 7687 7021 6871 6871
q18 12982 12138 12830 12138
q19 883 784 790 784
q20 2131 2125 1891 1891
q21 5558 4597 4630 4597
q22 480 476 409 409
Total cold run time: 71666 ms
Total hot run time: 65667 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6545417319
<details>
<summary>ClickBench: <b>Total hot run time: 25.06 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bec29aad984da153cab0fdcc267967ab66d9c543, data reload: false
query1 0.01 0.01 0.00
query2 0.14 0.10 0.09
query3 0.34 0.12 0.12
query4 1.61 0.14 0.13
query5 0.26 0.25 0.25
query6 1.15 0.64 0.64
query7 0.03 0.01 0.01
query8 0.10 0.07 0.08
query9 0.44 0.35 0.35
query10 0.58 0.61 0.59
query11 0.31 0.18 0.18
query12 0.32 0.19 0.19
query13 0.51 0.51 0.52
query14 0.96 0.93 0.93
query15 0.65 0.59 0.58
query16 0.35 0.35 0.35
query17 1.02 1.04 1.02
query18 0.28 0.26 0.27
query19 1.93 1.85 1.85
query20 0.01 0.01 0.02
query21 15.39 0.34 0.30
query22 4.93 0.13 0.13
query23 15.86 0.48 0.28
query24 1.96 0.53 0.39
query25 0.15 0.10 0.10
query26 0.76 0.28 0.21
query27 0.10 0.10 0.10
query28 3.47 0.77 0.39
query29 12.60 4.31 3.38
query30 0.33 0.16 0.14
query31 2.76 0.59 0.33
query32 3.22 0.58 0.47
query33 3.12 2.88 3.02
query34 15.88 4.05 3.46
query35 3.35 3.35 3.36
query36 0.60 0.51 0.50
query37 0.11 0.09 0.10
query38 0.07 0.07 0.06
query39 0.06 0.06 0.06
query40 0.21 0.17 0.16
query41 0.10 0.05 0.06
query42 0.05 0.06 0.05
query43 0.06 0.06 0.05
Total cold run time: 96.14 s
Total hot run time: 25.06 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](iceberg) Commit every Iceberg file under the table's own location [doris]
Posted by "hello-stephen (via GitHub) via commits" <co...@doris.apache.org>.
hello-stephen commented on PR #69392:
URL: https://github.com/apache/doris/pull/69392#issuecomment-6547057242
<details>
<summary>TPC-H: <b>Total hot run time: 28246 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c8d917291673e51cea99c571dd329aab21e2bf25, data reload: false
------ Round 1 ----------------------------------
============================================
q1 16626 5097 5033 5033
q2 2119 304 250 250
q3 9921 1372 812 812
q4 4606 456 331 331
q5 7427 812 530 530
q6 176 181 152 152
q7 753 789 590 590
q8 9369 1330 1055 1055
q9 5456 4163 4153 4153
q10 6811 1273 1021 1021
q11 425 234 221 221
q12 636 397 278 278
q13 18119 3662 2996 2996
q14 275 276 246 246
q15 q16 707 714 662 662
q17 1204 739 581 581
q18 6911 6198 6157 6157
q19 1098 950 599 599
q20 402 336 225 225
q21 5195 2100 2243 2100
q22 344 293 254 254
Total cold run time: 98580 ms
Total hot run time: 28246 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5552 5547 5579 5547
q2 674 557 538 538
q3 5229 5382 4992 4992
q4 1963 2115 1548 1548
q5 5242 5143 5130 5130
q6 243 259 228 228
q7 2425 2025 1856 1856
q8 2950 2680 2631 2631
q9 8069 7965 7904 7904
q10 3743 3756 3242 3242
q11 605 447 440 440
q12 684 709 523 523
q13 3541 3829 3338 3338
q14 326 312 308 308
q15 q16 700 731 632 632
q17 8214 6992 6878 6878
q18 12963 12199 12861 12199
q19 915 822 817 817
q20 2141 2152 1942 1942
q21 5535 4568 4625 4568
q22 495 458 403 403
Total cold run time: 72209 ms
Total hot run time: 65664 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org