You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zclllyybb (via GitHub)" <gi...@apache.org> on 2026/03/24 12:31:50 UTC
[PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
zclllyybb opened a new pull request, #61682:
URL: https://github.com/apache/doris/pull/61682
and split individual `CastToTimestampTz` class
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4124463068
<details>
<summary>TPC-H: <b>Total hot run time: 26914 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a4a59600771c3f9a9e56c66531570af4078551ce, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17612 4582 4290 4290
q2 q3 10641 795 530 530
q4 4679 352 247 247
q5 7554 1218 1036 1036
q6 176 178 148 148
q7 804 855 698 698
q8 9331 1496 1377 1377
q9 4935 4771 4772 4771
q10 6241 1915 1698 1698
q11 487 257 244 244
q12 694 596 478 478
q13 18060 2708 1953 1953
q14 234 235 212 212
q15 q16 739 739 703 703
q17 747 884 427 427
q18 6176 5579 5293 5293
q19 1106 986 649 649
q20 539 492 395 395
q21 4489 1858 1452 1452
q22 542 378 313 313
Total cold run time: 95786 ms
Total hot run time: 26914 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4797 4696 4582 4582
q2 q3 3914 4356 3866 3866
q4 888 1175 794 794
q5 4144 4426 4392 4392
q6 183 177 140 140
q7 1816 1702 1507 1507
q8 2512 2750 2588 2588
q9 7751 7413 7528 7413
q10 3885 4021 3609 3609
q11 528 426 417 417
q12 506 616 470 470
q13 2449 2974 2111 2111
q14 276 331 282 282
q15 q16 713 754 709 709
q17 1156 1331 1314 1314
q18 7054 6797 6777 6777
q19 936 939 929 929
q20 2124 2227 2039 2039
q21 4097 3457 3346 3346
q22 501 422 391 391
Total cold run time: 50230 ms
Total hot run time: 47676 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4117873373
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4123891726
## Code Review — No Blocking Issues Found
Reviewed the full diff (47 files, +1349/-425). This is a well-executed mechanical refactoring with several embedded bug fixes.
### Critical Checkpoint Conclusions
**1. Goal & Correctness**: The PR replaces legacy `from_date_str` methods and boolean template parameters (`IS_STRICT`, `IS_DATETIME`) with strongly-typed enums (`DatelikeParseMode`, `DatelikeTargetType`) and centralized cast functions. All replacement sites are correct — the new code preserves the original dispatch logic for every caller.
**2. Modification Scope**: Focused on the enum migration and API consolidation. Also fixes several pre-existing bugs:
- `from_decimal_batch` for DateV1/DateTimeV1 had `IS_STRICT=true` in the template but `params.is_strict=false` at runtime — now correctly uses `NON_STRICT`
- `from_decimal` slow-dispatch overload had `TimeValue::TimeType&` parameter type (wrong for Date/DateTime context) — now correctly uses `VecDateTimeValue&`
- `function_test_util.cpp` called `from_string_strict_mode<false>` with `is_strict=false` — template/runtime mismatch fixed to both use STRICT
- `FieldTypeTraits<TIMESTAMPTZ>::from_string` defaulted to non-strict — now correctly uses strict mode for OLAP field parsing
- Typo fix: `CastToTimstampTz` → `CastToTimestampTz`
**3. Concurrency**: N/A — all changes are in stateless static functions.
**4. Lifecycle Management**: N/A.
**5. Configuration Items**: None added.
**6. Incompatible Changes**: `from_date_str` removed from `VecDateTimeValue` and `DateV2Value<T>` public API. This is internal BE API only.
**7. Parallel Code Paths**: Enum migration applied uniformly across all datelike types (Date, DateTime, DateV2, DateTimeV2, TimeV2, TimestampTz). No paths missed.
**8. Special Conditional Checks**: `DCHECK(IsStrict == params.is_strict)` guards added to catch template/runtime parameter mismatches in debug builds — appropriate use of assertions.
**9. Test Coverage**: New `data_type_serde_datelike_batch_test.cpp` (454 lines) covers from_string, from_float, from_decimal batch operations for all types including mixed valid/invalid rows. The mixed-batch tests (`datev1_from_decimal_batch_mixed`, `datetimev1_from_decimal_batch_mixed`) serve as regression tests for the IS_STRICT bug fix. Existing test files updated to use new API.
**10. Observability**: N/A for this refactoring.
**11. Performance**: Zero runtime cost — enum-to-bool conversions are `constexpr`. The `static_cast<void>(to_scale)` in `CastToDateOrDatetime::from_float` is a no-op to suppress unused parameter warning after interface unification.
**12. Other Observations**:
- `CastToTimestampTz` properly extracted into its own file with `friend` access to `CastToDatetimeV2`'s private `_internal` methods
- The `_read_column_from_arrow` change correctly hardcodes `DATE_TIME` since `VecDateTimeValue()` defaults to `TIME_DATETIME`, matching old runtime dispatch behavior
- `read_date_text_impl` / `read_datetime_text_impl` correctly use `DATE` / `DATE_TIME` respectively, matching the old `from_date_str` runtime type check
- New `Int64` overloads in `io_helper.cpp` properly use `binary_cast` round-trip
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4117873298
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4125265078
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a4a59600771c3f9a9e56c66531570af4078551ce_a4a59600771c3f9a9e56c66531570af4078551ce/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a4a59600771c3f9a9e56c66531570af4078551ce_a4a59600771c3f9a9e56c66531570af4078551ce/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.86% (19887/37624) |
| Line Coverage | 36.37% (185933/511241) |
| Region Coverage | 32.60% (144007/441690) |
| Branch Coverage | 33.81% (63014/186401) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4150012358
<details>
<summary>TPC-DS: <b>Total hot run time: 169280 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 84d77460b7aef684ece6b1aa7d754d4ae30764a9, data reload: false
query5 4326 657 528 528
query6 332 225 205 205
query7 4195 472 273 273
query8 338 242 232 232
query9 8725 2765 2713 2713
query10 489 400 339 339
query11 6973 5123 4877 4877
query12 176 126 122 122
query13 1267 440 345 345
query14 5727 3666 3445 3445
query14_1 2852 2800 2781 2781
query15 204 190 172 172
query16 962 466 449 449
query17 914 724 628 628
query18 2445 458 367 367
query19 231 217 188 188
query20 132 127 138 127
query21 226 138 113 113
query22 13179 14063 14361 14063
query23 16812 16231 15973 15973
query23_1 16277 16363 15906 15906
query24 7200 1660 1237 1237
query24_1 1261 1234 1244 1234
query25 576 498 441 441
query26 1241 274 153 153
query27 2775 495 298 298
query28 4485 1875 1863 1863
query29 891 590 539 539
query30 309 227 195 195
query31 1003 964 881 881
query32 83 75 76 75
query33 519 352 297 297
query34 903 867 519 519
query35 650 693 614 614
query36 1066 1119 1000 1000
query37 146 97 89 89
query38 2921 2938 2892 2892
query39 847 843 828 828
query39_1 847 786 792 786
query40 240 152 132 132
query41 67 59 60 59
query42 266 259 259 259
query43 240 255 228 228
query44
query45 195 184 189 184
query46 875 986 612 612
query47 2140 2106 2063 2063
query48 322 308 223 223
query49 634 477 387 387
query50 696 288 218 218
query51 4035 4033 3998 3998
query52 266 268 257 257
query53 288 343 287 287
query54 297 284 262 262
query55 91 89 85 85
query56 324 327 316 316
query57 1932 1822 1746 1746
query58 278 270 274 270
query59 2772 2962 2747 2747
query60 345 333 328 328
query61 160 156 157 156
query62 665 600 544 544
query63 311 274 274 274
query64 5176 1305 993 993
query65
query66 1470 451 387 387
query67 24142 24238 24135 24135
query68
query69 409 324 308 308
query70 999 952 967 952
query71 336 315 294 294
query72 2860 2673 2490 2490
query73 540 541 312 312
query74 9628 9589 9382 9382
query75 2855 2899 2447 2447
query76 2279 1035 671 671
query77 352 376 317 317
query78 10901 11100 10441 10441
query79 3044 799 564 564
query80 1755 638 553 553
query81 587 275 223 223
query82 984 155 119 119
query83 330 273 242 242
query84 298 123 103 103
query85 946 515 464 464
query86 419 304 305 304
query87 3173 3114 2998 2998
query88 3549 2695 2662 2662
query89 416 375 339 339
query90 1976 183 177 177
query91 177 170 140 140
query92 79 74 73 73
query93 1371 859 511 511
query94 665 306 296 296
query95 600 353 382 353
query96 644 515 231 231
query97 2484 2518 2431 2431
query98 238 219 218 218
query99 1006 993 936 936
Total cold run time: 253122 ms
Total hot run time: 169280 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4118801512
<details>
<summary>TPC-DS: <b>Total hot run time: 169719 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 655e7aa97d6961878ec0da6ddffd929f26cc8bcb, data reload: false
query5 4313 633 506 506
query6 348 225 203 203
query7 4214 463 266 266
query8 337 253 219 219
query9 8686 2735 2744 2735
query10 470 390 369 369
query11 6980 5097 4871 4871
query12 187 134 126 126
query13 1284 466 384 384
query14 5758 3704 3454 3454
query14_1 2802 2765 2787 2765
query15 208 192 174 174
query16 1006 472 455 455
query17 1093 696 605 605
query18 2426 440 345 345
query19 239 212 184 184
query20 138 125 129 125
query21 213 131 106 106
query22 13250 14027 14495 14027
query23 17003 16445 16419 16419
query23_1 16231 16390 15771 15771
query24 7129 1617 1217 1217
query24_1 1248 1238 1232 1232
query25 573 484 449 449
query26 1257 267 159 159
query27 2770 497 303 303
query28 4512 1837 1867 1837
query29 889 581 491 491
query30 300 227 196 196
query31 1021 955 874 874
query32 87 76 72 72
query33 517 342 301 301
query34 913 871 525 525
query35 651 682 603 603
query36 1108 1130 996 996
query37 141 102 88 88
query38 2939 2933 2849 2849
query39 864 833 812 812
query39_1 802 789 799 789
query40 236 162 145 145
query41 72 68 69 68
query42 269 267 255 255
query43 247 256 231 231
query44
query45 210 189 236 189
query46 889 989 618 618
query47 2137 2124 2091 2091
query48 308 321 223 223
query49 625 478 384 384
query50 696 286 215 215
query51 4090 4104 3989 3989
query52 268 269 259 259
query53 298 335 282 282
query54 292 282 268 268
query55 90 86 83 83
query56 317 326 320 320
query57 1924 1814 1736 1736
query58 281 273 274 273
query59 2816 2938 2748 2748
query60 343 343 336 336
query61 153 145 147 145
query62 616 596 550 550
query63 314 277 276 276
query64 5013 1343 1043 1043
query65
query66 1471 449 357 357
query67 24131 24307 24265 24265
query68
query69 403 328 288 288
query70 999 1003 980 980
query71 347 314 307 307
query72 2866 2720 2472 2472
query73 545 548 321 321
query74 9624 9613 9486 9486
query75 2889 2761 2457 2457
query76 2285 1021 666 666
query77 369 392 314 314
query78 10984 11072 10513 10513
query79 3124 745 568 568
query80 1792 636 558 558
query81 549 258 222 222
query82 989 151 118 118
query83 350 264 245 245
query84 256 123 104 104
query85 929 501 455 455
query86 412 309 310 309
query87 3163 3104 3014 3014
query88 3575 2662 2643 2643
query89 430 386 344 344
query90 2015 184 178 178
query91 172 167 139 139
query92 81 76 71 71
query93 1220 848 498 498
query94 653 328 284 284
query95 596 411 325 325
query96 649 519 229 229
query97 2473 2511 2400 2400
query98 247 231 227 227
query99 1000 962 919 919
Total cold run time: 252889 ms
Total hot run time: 169719 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61682:
URL: https://github.com/apache/doris/pull/61682#discussion_r2981271196
##########
be/src/exprs/function/cast/cast_to_timestamptz_impl.hpp:
##########
@@ -0,0 +1,74 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "exprs/function/cast/cast_to_datetimev2_impl.hpp"
+
+namespace doris {
+
+/**
+ * CastToTimestampTz wraps CastToDatetimeV2 with DataTimeCastEnumType::TIMESTAMP_TZ hardcoded,
+ * so external callers never need to specify DataTimeCastEnumType explicitly.
+ *
+ * The difference from CastToDatetimeV2:
+ * - When timezone info is present in the string, TIMESTAMP_TZ converts to UTC;
+ * DATE_TIME converts to local_time_zone.
+ * - When timezone info is absent, TIMESTAMP_TZ treats input as local time and converts to UTC;
+ * DATE_TIME keeps the time as-is.
+ */
+struct CastToTimestampTz {
Review Comment:
**ODR concern:** This `struct CastToTimestampTz` has the same name as the one in `cast_to_timestamptz.h`, but they are completely different structs (different methods, different parameter types). While no TU currently includes both headers, this is a latent ODR violation — if any future code transitively includes both, it will be undefined behavior.
The `.h` version wraps `TimestampTzValue&` and delegates to `TimestampTzValue::from_string()`, which then calls this `.hpp` version's methods on `DateV2Value<DateTimeV2ValueType>&`.
Consider renaming this struct to something like `CastToTimestampTzInternal` or consolidating the two definitions to prevent future issues.
##########
be/src/runtime/runtime_state.cpp:
##########
@@ -204,11 +205,14 @@ Status RuntimeState::init(const TUniqueId& fragment_instance_id, const TQueryOpt
} else if (!query_globals.now_string.empty()) {
_timezone = TimezoneUtils::default_time_zone;
VecDateTimeValue dt;
- dt.from_date_str(query_globals.now_string.c_str(), query_globals.now_string.size());
+ CastParameters params;
Review Comment:
Nit: Extra leading space introduced on this line — `_nano_seconds` has 9 spaces of indentation instead of the expected 8. Should be:
```cpp
_nano_seconds = 0;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4119675126
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61682_655e7aa97d6961878ec0da6ddffd929f26cc8bcb_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61682_655e7aa97d6961878ec0da6ddffd929f26cc8bcb_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.32% (27007/36835) |
| Line Coverage | 56.82% (289605/509688) |
| Region Coverage | 54.18% (241383/445524) |
| Branch Coverage | 55.87% (104419/186902) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4149996605
<details>
<summary>TPC-H: <b>Total hot run time: 26743 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 84d77460b7aef684ece6b1aa7d754d4ae30764a9, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17619 4479 4271 4271
q2 q3 10639 787 526 526
q4 4679 364 257 257
q5 7578 1219 1017 1017
q6 176 172 143 143
q7 806 856 677 677
q8 10120 1499 1340 1340
q9 6460 4803 4751 4751
q10 6306 1947 1634 1634
q11 484 253 245 245
q12 700 597 462 462
q13 18040 2735 1960 1960
q14 235 232 221 221
q15 q16 738 741 674 674
q17 739 839 467 467
q18 6427 5414 5266 5266
q19 1108 1005 602 602
q20 532 494 369 369
q21 4363 2081 1576 1576
q22 378 345 285 285
Total cold run time: 98127 ms
Total hot run time: 26743 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4672 4582 4601 4582
q2 q3 3939 4496 3863 3863
q4 875 1199 788 788
q5 4063 4412 4333 4333
q6 188 178 140 140
q7 1801 1681 1553 1553
q8 2606 2721 2596 2596
q9 7602 7504 7343 7343
q10 3737 3997 3669 3669
q11 549 450 441 441
q12 504 601 453 453
q13 2858 2923 2065 2065
q14 292 310 276 276
q15 q16 723 800 713 713
q17 1179 1448 1387 1387
q18 7499 6855 6683 6683
q19 909 898 941 898
q20 2068 2223 1996 1996
q21 4039 3495 3362 3362
q22 467 433 391 391
Total cold run time: 50570 ms
Total hot run time: 47532 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4152114135
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4150336059
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61682_84d77460b7aef684ece6b1aa7d754d4ae30764a9_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61682_84d77460b7aef684ece6b1aa7d754d4ae30764a9_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.82% (26515/36917) |
| Line Coverage | 54.67% (279674/511601) |
| Region Coverage | 51.68% (231367/447697) |
| Branch Coverage | 53.21% (100109/188139) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4118721138
<details>
<summary>TPC-H: <b>Total hot run time: 26945 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 655e7aa97d6961878ec0da6ddffd929f26cc8bcb, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17630 4458 4274 4274
q2 q3 10635 810 541 541
q4 4688 344 251 251
q5 7642 1234 1012 1012
q6 173 174 145 145
q7 793 853 679 679
q8 9892 1473 1355 1355
q9 5608 4823 4784 4784
q10 6317 1965 1675 1675
q11 456 247 244 244
q12 760 583 465 465
q13 18032 2712 1958 1958
q14 225 233 222 222
q15 q16 758 753 663 663
q17 746 864 423 423
q18 5961 5484 5297 5297
q19 1138 984 605 605
q20 525 493 369 369
q21 4539 1849 1712 1712
q22 395 367 271 271
Total cold run time: 96913 ms
Total hot run time: 26945 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4758 4569 4579 4569
q2 q3 3880 4362 3857 3857
q4 893 1205 801 801
q5 4173 4552 4339 4339
q6 197 182 146 146
q7 1786 1679 1552 1552
q8 2606 2815 2593 2593
q9 7428 7507 7481 7481
q10 3746 4038 3564 3564
q11 505 445 418 418
q12 467 611 458 458
q13 2413 3220 2086 2086
q14 293 308 279 279
q15 q16 733 751 706 706
q17 1171 1395 1376 1376
q18 7457 6893 6690 6690
q19 910 895 918 895
q20 2102 2151 2036 2036
q21 3939 3573 3306 3306
q22 484 435 366 366
Total cold run time: 49941 ms
Total hot run time: 47518 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4119635827
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/655e7aa97d6961878ec0da6ddffd929f26cc8bcb_655e7aa97d6961878ec0da6ddffd929f26cc8bcb/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/655e7aa97d6961878ec0da6ddffd929f26cc8bcb_655e7aa97d6961878ec0da6ddffd929f26cc8bcb/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.79% (19858/37618) |
| Line Coverage | 36.29% (185529/511240) |
| Region Coverage | 32.54% (143606/441384) |
| Branch Coverage | 33.75% (62885/186334) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4125548913
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61682_a4a59600771c3f9a9e56c66531570af4078551ce_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61682_a4a59600771c3f9a9e56c66531570af4078551ce_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.81% (26455/36841) |
| Line Coverage | 54.66% (278578/509678) |
| Region Coverage | 52.01% (231866/445816) |
| Branch Coverage | 53.35% (99746/186965) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61682:
URL: https://github.com/apache/doris/pull/61682#discussion_r2985627761
##########
be/src/core/data_type_serde/data_type_date_or_datetime_serde.cpp:
##########
@@ -550,9 +571,12 @@ Status DataTypeDateSerDe<T>::from_decimal_batch(
CastParameters params {.status = Status::OK(), .is_strict = false};
for (size_t i = 0; i < decimal_col.size(); ++i) {
CppType val;
- if (CastToDateOrDatetime::from_decimal<true, IsDatetime>(
- decimal_col.get_intergral_part(i), decimal_col.get_fractional_part(i),
- decimal_col.get_scale(), val, params)) [[likely]] {
+ if (CastToDateOrDatetime::from_decimal < DatelikeParseMode::STRICT,
Review Comment:
**Inconsistency**: This `from_decimal_batch` (non-strict path, `params.is_strict = false`) uses `DatelikeParseMode::STRICT`, while the equivalent functions in `data_type_datev2_serde.cpp:421` and `data_type_datetimev2_serde.cpp:272` use `DatelikeParseMode::NON_STRICT`.
The old code in all three files used `<true>` (strict) in the non-strict batch, so for v1 date/datetime the old behavior is preserved. However, the v2 serde files were intentionally changed to `NON_STRICT` to match the semantic intent (`params.is_strict = false`). This v1 file should likely follow the same correction for consistency.
The practical impact is minor — in the non-strict batch path, `params.status` is not checked (only the return value determines nullability), so using STRICT just does unnecessary work writing an error message to `params.status` that nobody reads. But for consistency and clarity, this should be `DatelikeParseMode::NON_STRICT` to match the other two serde files.
Suggested fix:
```cpp
if (CastToDateOrDatetime::from_decimal < DatelikeParseMode::NON_STRICT,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4123026470
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4124514981
<details>
<summary>TPC-DS: <b>Total hot run time: 169177 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a4a59600771c3f9a9e56c66531570af4078551ce, data reload: false
query5 4327 664 526 526
query6 340 235 213 213
query7 4209 483 267 267
query8 354 250 226 226
query9 8722 2762 2759 2759
query10 518 406 351 351
query11 6998 5092 4882 4882
query12 191 128 128 128
query13 1279 466 354 354
query14 5783 3768 3419 3419
query14_1 2876 2785 2811 2785
query15 203 194 184 184
query16 991 472 480 472
query17 903 750 632 632
query18 2457 463 362 362
query19 220 221 185 185
query20 134 125 132 125
query21 215 137 114 114
query22 13214 14108 14848 14108
query23 16831 16353 15950 15950
query23_1 15943 15741 15662 15662
query24 7113 1614 1219 1219
query24_1 1237 1245 1222 1222
query25 564 467 406 406
query26 1247 258 147 147
query27 2792 484 293 293
query28 4471 1825 1835 1825
query29 849 559 473 473
query30 299 230 188 188
query31 1007 950 869 869
query32 84 76 70 70
query33 517 341 292 292
query34 910 882 531 531
query35 648 675 621 621
query36 1079 1164 930 930
query37 127 94 86 86
query38 2942 2933 2871 2871
query39 848 844 813 813
query39_1 798 784 789 784
query40 231 151 138 138
query41 63 60 60 60
query42 263 261 257 257
query43 249 245 224 224
query44
query45 197 191 185 185
query46 889 1002 613 613
query47 2149 2143 2023 2023
query48 327 327 246 246
query49 641 462 387 387
query50 688 283 221 221
query51 4077 4071 3983 3983
query52 262 270 258 258
query53 287 340 289 289
query54 311 304 286 286
query55 92 86 85 85
query56 315 325 306 306
query57 1941 1790 1789 1789
query58 291 277 278 277
query59 2826 2965 2738 2738
query60 341 348 329 329
query61 157 149 156 149
query62 631 597 540 540
query63 309 285 273 273
query64 5135 1288 1044 1044
query65
query66 1461 456 375 375
query67 24324 24370 24230 24230
query68
query69 404 312 284 284
query70 989 1009 967 967
query71 356 307 330 307
query72 2855 2733 2519 2519
query73 532 547 311 311
query74 9646 9600 9421 9421
query75 2849 2777 2470 2470
query76 2280 1028 685 685
query77 366 389 319 319
query78 10899 11088 10477 10477
query79 1065 844 568 568
query80 794 629 567 567
query81 491 268 222 222
query82 1308 159 121 121
query83 340 269 252 252
query84 292 124 109 109
query85 902 499 452 452
query86 386 318 274 274
query87 3153 3117 3020 3020
query88 3584 2687 2674 2674
query89 426 367 346 346
query90 1927 184 177 177
query91 170 163 136 136
query92 75 78 71 71
query93 911 837 532 532
query94 463 312 296 296
query95 604 350 331 331
query96 658 520 225 225
query97 2470 2495 2388 2388
query98 245 227 215 215
query99 1002 1012 925 925
Total cold run time: 249594 ms
Total hot run time: 169177 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4150194652
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/84d77460b7aef684ece6b1aa7d754d4ae30764a9_84d77460b7aef684ece6b1aa7d754d4ae30764a9/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/84d77460b7aef684ece6b1aa7d754d4ae30764a9_84d77460b7aef684ece6b1aa7d754d4ae30764a9/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.93% (19951/37695) |
| Line Coverage | 36.46% (187071/513139) |
| Region Coverage | 32.69% (144991/443575) |
| Branch Coverage | 33.89% (63560/187573) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb merged PR #61682:
URL: https://github.com/apache/doris/pull/61682
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4117873607
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4149822850
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4152759152
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4123841881
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](datetime) Replace legacy `from_date_str` with cast function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #61682:
URL: https://github.com/apache/doris/pull/61682#issuecomment-4124040398
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org