You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "seawinde (via GitHub)" <gi...@apache.org> on 2026/05/13 03:22:32 UTC
[PR] [feature](function) Support murmur_hash3_128 function [doris]
seawinde opened a new pull request, #63196:
URL: https://github.com/apache/doris/pull/63196
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
This PR adds a builtin 128-bit MurmurHash3 scalar function, `murmur_hash3_128`, for callers that need a wider hash value than the existing 32-bit and 64-bit variants.
| File | Change Description |
|------|--------------------|
| `be/src/exprs/function/function_hash.cpp` | Adds BE implementation for `murmur_hash3_128`, returning LARGEINT. |
| `be/test/exprs/function/function_hash_test.cpp` | Adds unit coverage for constant and multi-argument hash cases. |
| `BuiltinScalarFunctions.java` | Registers the Nereids scalar function. |
| `MurmurHash3128.java` | Adds FE scalar function metadata and signatures. |
| `ScalarFunctionVisitor.java` | Adds visitor entry for the new scalar function. |
Design rationale: the function reuses the existing MurmurHash3 x64 128-bit processing path in BE and exposes the packed 128-bit result as LARGEINT, matching Doris' existing signed 128-bit integer representation.
### Release note
Support `murmur_hash3_128` function.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Unit tests run locally:
- `./run-fe-ut.sh --coverage --run org.apache.doris.nereids.rules.analysis.FunctionRegistryTest` passed.
- `./run-be-ut.sh --run '--filter=HashFunctionTest.*'` was attempted, but the local worktree failed before compilation while downloading the `apache-orc` submodule from GitHub.
- Behavior changed:
- [ ] No.
- [x] Yes. Adds a new builtin scalar function.
- Does this need documentation?
- [ ] No.
- [x] Yes. Function documentation should be added if this PR is kept beyond CI validation.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4448259834
# BE Regression && UT Coverage Report
Increment line coverage `75.81% (47/62)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.80% (27830/37712) |
| Line Coverage | 57.73% (301875/522934) |
| Region Coverage | 55.04% (252399/458594) |
| Branch Coverage | 56.53% (109105/193005) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4448035444
# BE UT Coverage Report
Increment line coverage `75.81% (47/62)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c79471f90a154f510ebb6a6e7b3f8b314da95e4d_c79471f90a154f510ebb6a6e7b3f8b314da95e4d/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c79471f90a154f510ebb6a6e7b3f8b314da95e4d_c79471f90a154f510ebb6a6e7b3f8b314da95e4d/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.58% (20635/38513) |
| Line Coverage | 37.20% (195078/524350) |
| Region Coverage | 33.61% (152683/454289) |
| Branch Coverage | 34.60% (66543/192303) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586240545
# FE UT Coverage Report
Increment line coverage ` 6.67% (2/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586472905
<details>
<summary>TPC-DS: <b>Total hot run time: 172059 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4, data reload: false
query5 4319 674 515 515
query6 338 214 199 199
query7 4297 613 303 303
query8 330 229 227 227
query9 8788 3979 4012 3979
query10 475 355 314 314
query11 5827 2440 2297 2297
query12 177 130 124 124
query13 1289 613 444 444
query14 6109 5485 5228 5228
query14_1 4482 4478 4437 4437
query15 218 211 181 181
query16 993 476 390 390
query17 987 763 616 616
query18 2458 524 364 364
query19 232 212 171 171
query20 134 129 128 128
query21 221 149 119 119
query22 13720 13534 13334 13334
query23 17440 16671 16305 16305
query23_1 16325 16430 16245 16245
query24 7786 1798 1337 1337
query24_1 1352 1326 1321 1321
query25 577 508 451 451
query26 1307 347 182 182
query27 2672 584 344 344
query28 4463 2047 2006 2006
query29 1025 657 526 526
query30 316 239 206 206
query31 1135 1077 966 966
query32 90 76 76 76
query33 559 364 310 310
query34 1218 1193 667 667
query35 775 832 701 701
query36 1387 1381 1250 1250
query37 158 102 89 89
query38 3253 3197 3086 3086
query39 910 920 922 920
query39_1 872 880 894 880
query40 248 163 138 138
query41 78 72 76 72
query42 116 113 108 108
query43 338 329 290 290
query44
query45 217 207 194 194
query46 1100 1230 735 735
query47 2401 2392 2221 2221
query48 419 460 310 310
query49 632 492 375 375
query50 962 354 254 254
query51 4375 4306 4239 4239
query52 107 106 104 104
query53 260 275 216 216
query54 309 271 265 265
query55 92 90 85 85
query56 288 295 298 295
query57 1441 1413 1294 1294
query58 300 265 267 265
query59 1615 1646 1429 1429
query60 326 344 317 317
query61 161 158 150 150
query62 704 655 586 586
query63 242 202 203 202
query64 2408 816 645 645
query65
query66 1765 475 358 358
query67 29847 29752 29645 29645
query68
query69 463 339 306 306
query70 1071 1015 996 996
query71 312 274 262 262
query72 3058 2742 2515 2515
query73 895 799 427 427
query74 5130 4932 4841 4841
query75 2696 2633 2288 2288
query76 2287 1177 775 775
query77 398 419 346 346
query78 12506 12480 11881 11881
query79 1495 1041 785 785
query80 640 536 447 447
query81 455 281 237 237
query82 1388 166 127 127
query83 353 292 254 254
query84 303 151 114 114
query85 886 569 449 449
query86 406 343 300 300
query87 3428 3404 3257 3257
query88 3684 2741 2780 2741
query89 444 399 343 343
query90 1974 184 183 183
query91 182 174 140 140
query92 80 78 78 78
query93 1496 1503 820 820
query94 564 311 318 311
query95 666 402 349 349
query96 1069 780 355 355
query97 2759 2734 2589 2589
query98 236 237 228 228
query99 1188 1150 1042 1042
Total cold run time: 255095 ms
Total hot run time: 172059 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4447024135
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on code in PR #63196:
URL: https://github.com/apache/doris/pull/63196#discussion_r3257951288
##########
be/src/exprs/function/function_hash.cpp:
##########
@@ -133,6 +133,107 @@ using FunctionMurmurHash3_64_V2 =
using FunctionMurmurHash3U64V2 =
FunctionVariadicArgumentsBase<DataTypeInt128, MurmurHash3Impl<TYPE_LARGEINT, true>>;
+struct MurmurHash3128Impl {
+ static constexpr auto name = "murmur_hash3_128";
+
+ static Status empty_apply(IColumn& icolumn, size_t input_rows_count) {
+ ColumnVector<TYPE_LARGEINT>& vec_to = assert_cast<ColumnVector<TYPE_LARGEINT>&>(icolumn);
+ vec_to.get_data().assign(input_rows_count, pack_hash(emtpy_value, emtpy_value));
+ return Status::OK();
+ }
+
+ static Status first_apply(const IDataType* type, const IColumn* column, size_t input_rows_count,
+ IColumn& icolumn) {
+ return execute<true>(type, column, input_rows_count, icolumn);
+ }
+
+ static Status combine_apply(const IDataType* type, const IColumn* column,
+ size_t input_rows_count, IColumn& icolumn) {
+ return execute<false>(type, column, input_rows_count, icolumn);
+ }
+
+ template <bool first>
+ static Status execute(const IDataType* type, const IColumn* column, size_t input_rows_count,
+ IColumn& col_to) {
+ auto& to_column = assert_cast<ColumnVector<TYPE_LARGEINT>&>(col_to);
+ if constexpr (first) {
+ // The first argument initializes one 128-bit hash state per row. Later arguments reuse
+ // the same result column and update the saved state in place.
+ to_column.insert_many_defaults(input_rows_count);
+ }
+ auto& col_to_data = to_column.get_data();
+ if (const auto* col_from = check_and_get_column<ColumnString>(column)) {
+ const typename ColumnString::Chars& data = col_from->get_chars();
+ const typename ColumnString::Offsets& offsets = col_from->get_offsets();
+ size_t size = offsets.size();
+ ColumnString::Offset current_offset = 0;
+ for (size_t i = 0; i < size; ++i) {
+ if constexpr (first) {
+ init_hash(col_to_data[i], reinterpret_cast<const char*>(&data[current_offset]),
+ offsets[i] - current_offset);
+ } else {
+ update_hash(col_to_data[i],
+ reinterpret_cast<const char*>(&data[current_offset]),
+ offsets[i] - current_offset);
+ }
+ current_offset = offsets[i];
+ }
+ } else if (const ColumnConst* col_from_const =
+ check_and_get_column_const_string_or_fixedstring(column)) {
+ auto value = col_from_const->get_value<TYPE_STRING>();
+ for (size_t i = 0; i < input_rows_count; ++i) {
+ if constexpr (first) {
+ init_hash(col_to_data[i], value.data(), value.size());
+ } else {
+ update_hash(col_to_data[i], value.data(), value.size());
+ }
+ }
+ } else {
+ DCHECK(false);
+ return Status::NotSupported("Illegal column {} of argument of function {}",
+ column->get_name(), name);
+ }
+ return Status::OK();
+ }
+
+private:
+ static __int128_t pack_hash(uint64_t h1, uint64_t h2) {
+ static_assert(sizeof(__int128_t) == sizeof(uint64_t) * 2);
+ // Store the two MurmurHash3 x64 128-bit lanes in a single LARGEINT value. Keep h1 in the
+ // low 64 bits and h2 in the high 64 bits to match murmur_hash3_x64_128's out[0]/out[1].
+ const auto value =
+ (static_cast<unsigned __int128>(h2) << 64) | static_cast<unsigned __int128>(h1);
+ return static_cast<__int128_t>(value);
Review Comment:
[python ](https://mmh3.readthedocs.io/en/stable/api.html#mmh3.hash128) 和 [go](https://github.com/twmb/murmur3/blob/v1.1.8/murmur128.go)的默认实现都是无符号的，这里可能得跟 mmh3_64一样，加一个`murmur_hash3_u128`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586600463
# FE Regression Coverage Report
Increment line coverage ` 93.33% (28/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4587013714
# BE Regression && UT Coverage Report
Increment line coverage `94.78% (109/115)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.10% (27492/38132) |
| Line Coverage | 55.88% (295721/529214) |
| Region Coverage | 52.69% (245321/465563) |
| Branch Coverage | 54.15% (106005/195744) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4598150399
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4436867248
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4573619329
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb merged PR #63196:
URL: https://github.com/apache/doris/pull/63196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459856858
# BE Regression && UT Coverage Report
Increment line coverage `88.00% (66/75)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.60% (27818/37795) |
| Line Coverage | 57.53% (301340/523761) |
| Region Coverage | 54.78% (251914/459861) |
| Branch Coverage | 56.30% (108868/193357) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459919358
# FE Regression Coverage Report
Increment line coverage ` 93.33% (14/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586065302
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4574752893
<details>
<summary>TPC-H: <b>Total hot run time: 31683 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3c305497fbefe4fad3e74d6a6cd5ea3a348873c1, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17647 3986 3935 3935
q2 q3 10780 1477 845 845
q4 4684 486 342 342
q5 7615 2333 2177 2177
q6 240 185 142 142
q7 939 801 637 637
q8 9361 1718 1608 1608
q9 5415 4962 4959 4959
q10 6401 2191 1884 1884
q11 456 273 248 248
q12 666 426 300 300
q13 18150 3394 2817 2817
q14 267 262 240 240
q15 q16 824 782 714 714
q17 969 982 935 935
q18 6910 5784 5512 5512
q19 1196 1260 1097 1097
q20 673 456 281 281
q21 5944 2914 2645 2645
q22 456 365 492 365
Total cold run time: 99593 ms
Total hot run time: 31683 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4884 4794 4858 4794
q2 q3 5066 5306 4596 4596
q4 2118 2190 1390 1390
q5 4836 4950 4792 4792
q6 231 178 130 130
q7 1906 1835 1584 1584
q8 2426 2228 2135 2135
q9 8079 7974 7504 7504
q10 4753 4721 4206 4206
q11 540 417 363 363
q12 740 737 525 525
q13 3080 3357 2793 2793
q14 272 290 260 260
q15 q16 675 693 610 610
q17 1289 1254 1269 1254
q18 7271 6958 6996 6958
q19 1115 1080 1125 1080
q20 2232 2221 1941 1941
q21 5279 4600 4446 4446
q22 529 454 431 431
Total cold run time: 57321 ms
Total hot run time: 51792 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4437252868
<details>
<summary>TPC-H: <b>Total hot run time: 29748 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17612 3854 3838 3838
q2 q3 10705 893 609 609
q4 4664 471 358 358
q5 7578 1336 1167 1167
q6 267 174 144 144
q7 976 952 755 755
q8 10351 1415 1283 1283
q9 6480 5417 5375 5375
q10 6326 2083 1808 1808
q11 473 270 255 255
q12 691 413 299 299
q13 18175 3351 2779 2779
q14 296 284 264 264
q15 q16 905 870 792 792
q17 1025 952 847 847
q18 6355 5706 5621 5621
q19 1616 1238 1100 1100
q20 510 409 267 267
q21 4529 2276 1884 1884
q22 425 352 303 303
Total cold run time: 99959 ms
Total hot run time: 29748 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4218 4097 4074 4074
q2 q3 4635 4796 4153 4153
q4 2096 2160 1389 1389
q5 4955 4974 5236 4974
q6 192 167 133 133
q7 2023 2025 1738 1738
q8 3469 3186 3207 3186
q9 8552 8480 8405 8405
q10 4501 4510 4269 4269
q11 621 429 422 422
q12 705 772 508 508
q13 3195 3632 3039 3039
q14 316 311 265 265
q15 q16 772 798 719 719
q17 1352 1384 1320 1320
q18 8090 7136 7144 7136
q19 1182 1225 1143 1143
q20 2249 2234 1973 1973
q21 6193 5432 4917 4917
q22 571 530 411 411
Total cold run time: 59887 ms
Total hot run time: 54174 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459219479
<details>
<summary>TPC-DS: <b>Total hot run time: 169194 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 76fb519e146749e3726ca0440c7c35d1094a27d8, data reload: false
query5 4350 647 530 530
query6 344 215 199 199
query7 4221 577 294 294
query8 317 223 237 223
query9 8860 4012 3985 3985
query10 446 332 306 306
query11 5816 2370 2195 2195
query12 183 130 126 126
query13 1290 637 415 415
query14 6011 5385 5039 5039
query14_1 4340 4383 4342 4342
query15 215 211 190 190
query16 1020 466 435 435
query17 1144 729 604 604
query18 2598 498 368 368
query19 217 210 178 178
query20 141 134 134 134
query21 217 141 126 126
query22 13560 13510 13365 13365
query23 17242 16333 16017 16017
query23_1 16220 16091 16061 16061
query24 7430 1737 1310 1310
query24_1 1320 1296 1315 1296
query25 561 504 451 451
query26 1328 331 181 181
query27 2668 562 347 347
query28 4444 1972 1937 1937
query29 1018 652 539 539
query30 313 246 203 203
query31 1123 1065 931 931
query32 93 78 74 74
query33 546 363 302 302
query34 1193 1124 651 651
query35 772 792 702 702
query36 1366 1361 1147 1147
query37 152 105 88 88
query38 3235 3107 3052 3052
query39 919 912 902 902
query39_1 894 881 892 881
query40 236 144 126 126
query41 69 66 63 63
query42 110 108 113 108
query43 320 328 281 281
query44
query45 211 198 191 191
query46 1042 1178 729 729
query47 2325 2337 2189 2189
query48 397 431 294 294
query49 639 483 388 388
query50 1036 354 248 248
query51 4335 4333 4255 4255
query52 100 106 99 99
query53 268 303 212 212
query54 310 275 248 248
query55 98 96 90 90
query56 289 299 306 299
query57 1400 1378 1322 1322
query58 302 273 273 273
query59 1564 1590 1380 1380
query60 315 321 307 307
query61 151 157 156 156
query62 679 622 561 561
query63 239 194 205 194
query64 2391 790 658 658
query65
query66 1740 473 346 346
query67 29249 29837 29818 29818
query68
query69 476 335 325 325
query70 1053 995 980 980
query71 314 268 268 268
query72 3005 2717 2421 2421
query73 845 782 407 407
query74 5068 4863 4747 4747
query75 2666 2579 2289 2289
query76 2305 1135 734 734
query77 401 407 339 339
query78 12189 12185 11631 11631
query79 1484 1074 774 774
query80 1293 531 460 460
query81 524 279 241 241
query82 980 162 119 119
query83 321 275 250 250
query84 263 135 110 110
query85 929 546 450 450
query86 450 341 346 341
query87 3436 3400 3196 3196
query88 3574 2670 2654 2654
query89 446 383 339 339
query90 1888 183 179 179
query91 179 175 136 136
query92 75 83 73 73
query93 1611 1433 832 832
query94 700 349 272 272
query95 662 384 438 384
query96 1106 802 322 322
query97 2704 2693 2547 2547
query98 232 233 229 229
query99 1112 1114 1000 1000
Total cold run time: 253247 ms
Total hot run time: 169194 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459155655
<details>
<summary>TPC-H: <b>Total hot run time: 30812 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 76fb519e146749e3726ca0440c7c35d1094a27d8, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17731 3912 3870 3870
q2 q3 10764 1352 812 812
q4 4682 472 352 352
q5 7614 2256 2121 2121
q6 260 173 135 135
q7 903 776 635 635
q8 9377 1692 1556 1556
q9 6355 4878 4886 4878
q10 6449 2105 1759 1759
q11 434 268 245 245
q12 702 428 296 296
q13 18173 3368 2766 2766
q14 258 255 227 227
q15 q16 811 780 708 708
q17 996 1010 1013 1010
q18 6635 5763 5378 5378
q19 1254 1277 1078 1078
q20 506 386 263 263
q21 5626 2551 2427 2427
q22 424 351 296 296
Total cold run time: 99954 ms
Total hot run time: 30812 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4198 4112 4108 4108
q2 q3 4488 4835 4367 4367
q4 2134 2178 1375 1375
q5 4391 4250 4252 4250
q6 231 174 129 129
q7 1784 1913 1643 1643
q8 2489 2101 2063 2063
q9 7715 7783 7699 7699
q10 4537 4499 4067 4067
q11 583 417 390 390
q12 730 759 522 522
q13 3426 3673 2946 2946
q14 305 302 260 260
q15 q16 753 743 685 685
q17 1346 1296 1315 1296
q18 7811 7322 6925 6925
q19 1132 1079 1099 1079
q20 2182 2190 1916 1916
q21 5321 4584 4459 4459
q22 510 453 394 394
Total cold run time: 56066 ms
Total hot run time: 50573 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4470693680
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4587034189
# FE Regression Coverage Report
Increment line coverage ` 93.33% (28/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586450169
<details>
<summary>TPC-H: <b>Total hot run time: 31641 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17778 4077 4068 4068
q2 q3 10783 1445 822 822
q4 4682 476 356 356
q5 7585 2316 2172 2172
q6 247 176 139 139
q7 952 775 660 660
q8 9430 1736 1656 1656
q9 5303 5021 4967 4967
q10 6412 2216 1861 1861
q11 453 278 253 253
q12 652 427 302 302
q13 18146 3455 2812 2812
q14 264 264 241 241
q15 q16 826 785 701 701
q17 904 967 908 908
q18 7086 5765 5644 5644
q19 1172 1223 1099 1099
q20 526 420 269 269
q21 5677 2660 2402 2402
q22 442 365 309 309
Total cold run time: 99320 ms
Total hot run time: 31641 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4375 4335 4337 4335
q2 q3 4569 4990 4362 4362
q4 2129 2239 1380 1380
q5 4469 4337 4345 4337
q6 309 259 160 160
q7 2121 2000 1668 1668
q8 2526 2263 2198 2198
q9 8419 7938 8094 7938
q10 4873 4826 4371 4371
q11 614 412 386 386
q12 773 786 606 606
q13 3259 3605 2987 2987
q14 291 311 283 283
q15 q16 746 734 640 640
q17 1350 1461 1342 1342
q18 7959 7472 7367 7367
q19 1212 1113 1155 1113
q20 2232 2219 1935 1935
q21 5298 4593 4490 4490
q22 519 461 415 415
Total cold run time: 58043 ms
Total hot run time: 52313 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4586585430
# BE Regression && UT Coverage Report
Increment line coverage `94.78% (109/115)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_42b2746b4b4e5c1fbdc8d9795c96d485eb1a0bd4_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.10% (27492/38132) |
| Line Coverage | 55.88% (295721/529214) |
| Region Coverage | 52.69% (245321/465563) |
| Branch Coverage | 54.15% (106005/195744) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4438103663
# FE Regression Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4447486713
<details>
<summary>TPC-H: <b>Total hot run time: 29625 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c79471f90a154f510ebb6a6e7b3f8b314da95e4d, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17667 3920 3870 3870
q2 q3 10697 890 612 612
q4 4698 469 351 351
q5 7785 1354 1137 1137
q6 272 173 141 141
q7 926 953 762 762
q8 10140 1437 1317 1317
q9 6650 5487 5313 5313
q10 6316 2063 1814 1814
q11 469 265 256 256
q12 689 433 299 299
q13 18157 3311 2786 2786
q14 290 287 264 264
q15 q16 909 854 791 791
q17 988 1129 679 679
q18 6516 5719 5559 5559
q19 1321 1209 996 996
q20 520 412 401 401
q21 4690 2324 1911 1911
q22 472 394 366 366
Total cold run time: 100172 ms
Total hot run time: 29625 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4840 4590 4979 4590
q2 q3 4688 4821 4206 4206
q4 2182 2242 1388 1388
q5 4981 5004 5308 5004
q6 198 180 140 140
q7 2042 1834 1634 1634
q8 3390 3125 3189 3125
q9 8589 8488 8408 8408
q10 4479 4526 4260 4260
q11 612 434 406 406
q12 694 746 511 511
q13 3277 3639 2921 2921
q14 311 304 274 274
q15 q16 764 785 741 741
q17 1404 1391 1281 1281
q18 8011 7149 7303 7149
q19 1183 1196 1170 1170
q20 2284 2238 1945 1945
q21 6164 5397 4826 4826
q22 522 480 393 393
Total cold run time: 60615 ms
Total hot run time: 54372 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4591666597
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459006929
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4437318147
<details>
<summary>TPC-DS: <b>Total hot run time: 170678 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4, data reload: false
query5 4315 664 509 509
query6 318 218 200 200
query7 4224 564 320 320
query8 341 235 212 212
query9 8831 4040 4031 4031
query10 457 361 310 310
query11 5984 2378 2194 2194
query12 186 151 133 133
query13 1280 644 447 447
query14 6810 5427 5051 5051
query14_1 4357 4320 4350 4320
query15 210 199 180 180
query16 995 460 430 430
query17 1230 767 625 625
query18 2729 501 351 351
query19 241 201 169 169
query20 145 131 135 131
query21 231 140 115 115
query22 13684 13961 14494 13961
query23 17252 16516 16155 16155
query23_1 16216 16336 16247 16247
query24 7772 1989 1412 1412
query24_1 1381 1366 1355 1355
query25 583 528 452 452
query26 1291 316 177 177
query27 2672 606 337 337
query28 4403 1965 1966 1965
query29 1016 664 548 548
query30 307 228 199 199
query31 1128 1078 931 931
query32 83 74 75 74
query33 568 358 308 308
query34 1166 1109 643 643
query35 784 779 675 675
query36 1334 1325 1173 1173
query37 149 97 93 93
query38 3207 3121 3063 3063
query39 924 917 907 907
query39_1 871 878 870 870
query40 233 159 140 140
query41 68 67 63 63
query42 109 109 109 109
query43 321 333 284 284
query44
query45 209 202 193 193
query46 1126 1187 751 751
query47 2273 2277 2173 2173
query48 406 406 294 294
query49 656 595 446 446
query50 703 274 223 223
query51 4370 4306 4153 4153
query52 106 111 98 98
query53 247 272 204 204
query54 306 278 245 245
query55 91 88 83 83
query56 302 314 293 293
query57 1408 1369 1301 1301
query58 295 271 266 266
query59 1548 1596 1369 1369
query60 343 337 319 319
query61 159 153 153 153
query62 670 609 555 555
query63 243 204 197 197
query64 2317 816 689 689
query65
query66 1686 501 397 397
query67 29412 29931 29830 29830
query68
query69 456 341 311 311
query70 1031 974 995 974
query71 347 284 265 265
query72 3061 2724 2235 2235
query73 857 718 405 405
query74 5117 4947 4790 4790
query75 2784 2665 2331 2331
query76 2301 1148 799 799
query77 416 424 343 343
query78 12971 13098 12405 12405
query79 1477 992 697 697
query80 1380 586 481 481
query81 521 278 237 237
query82 1008 157 124 124
query83 319 279 250 250
query84 263 138 118 118
query85 941 517 460 460
query86 434 317 324 317
query87 3418 3354 3202 3202
query88 3522 2670 2622 2622
query89 439 377 343 343
query90 1908 182 177 177
query91 178 165 138 138
query92 79 73 77 73
query93 1124 939 582 582
query94 718 339 298 298
query95 687 452 352 352
query96 1085 793 329 329
query97 2694 2667 2587 2587
query98 242 227 229 227
query99 1088 1120 964 964
Total cold run time: 255088 ms
Total hot run time: 170678 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4436867570
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4569983077
<details>
<summary>TPC-H: <b>Total hot run time: 31361 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 76fb519e146749e3726ca0440c7c35d1094a27d8, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17727 3953 3931 3931
q2 q3 10781 1373 819 819
q4 4693 473 347 347
q5 7703 2225 2077 2077
q6 396 178 137 137
q7 918 776 661 661
q8 9398 1644 1544 1544
q9 6938 4978 4947 4947
q10 6592 2219 1872 1872
q11 459 279 239 239
q12 693 424 297 297
q13 18234 3351 2825 2825
q14 271 262 234 234
q15 q16 830 789 704 704
q17 968 947 902 902
q18 6937 5703 5693 5693
q19 1210 1295 1116 1116
q20 527 405 269 269
q21 5701 2606 2451 2451
q22 435 358 296 296
Total cold run time: 101411 ms
Total hot run time: 31361 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4345 4267 4243 4243
q2 q3 4543 4923 4347 4347
q4 2070 2163 1371 1371
q5 4400 4268 4756 4268
q6 256 196 161 161
q7 1998 1842 1541 1541
q8 2458 2100 2136 2100
q9 7914 7863 8004 7863
q10 4792 4733 4301 4301
q11 771 419 378 378
q12 729 743 530 530
q13 3353 3522 2941 2941
q14 306 307 272 272
q15 q16 713 734 659 659
q17 1347 1327 1305 1305
q18 7944 7334 7077 7077
q19 1146 1103 1110 1103
q20 2213 2213 1951 1951
q21 5215 4537 4403 4403
q22 514 450 406 406
Total cold run time: 57027 ms
Total hot run time: 51220 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4576122813
# FE Regression Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4574508807
# FE UT Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #63196:
URL: https://github.com/apache/doris/pull/63196#discussion_r3254653038
##########
be/src/exprs/function/function_hash.cpp:
##########
@@ -133,6 +133,107 @@ using FunctionMurmurHash3_64_V2 =
using FunctionMurmurHash3U64V2 =
FunctionVariadicArgumentsBase<DataTypeInt128, MurmurHash3Impl<TYPE_LARGEINT, true>>;
+struct MurmurHash3128Impl {
+ static constexpr auto name = "murmur_hash3_128";
+
+ static Status empty_apply(IColumn& icolumn, size_t input_rows_count) {
+ ColumnVector<TYPE_LARGEINT>& vec_to = assert_cast<ColumnVector<TYPE_LARGEINT>&>(icolumn);
+ vec_to.get_data().assign(input_rows_count, pack_hash(emtpy_value, emtpy_value));
+ return Status::OK();
+ }
+
+ static Status first_apply(const IDataType* type, const IColumn* column, size_t input_rows_count,
+ IColumn& icolumn) {
+ return execute<true>(type, column, input_rows_count, icolumn);
+ }
+
+ static Status combine_apply(const IDataType* type, const IColumn* column,
+ size_t input_rows_count, IColumn& icolumn) {
+ return execute<false>(type, column, input_rows_count, icolumn);
+ }
+
+ template <bool first>
+ static Status execute(const IDataType* type, const IColumn* column, size_t input_rows_count,
+ IColumn& col_to) {
+ auto& to_column = assert_cast<ColumnVector<TYPE_LARGEINT>&>(col_to);
+ if constexpr (first) {
+ // The first argument initializes one 128-bit hash state per row. Later arguments reuse
+ // the same result column and update the saved state in place.
+ to_column.insert_many_defaults(input_rows_count);
+ }
+ auto& col_to_data = to_column.get_data();
+ if (const auto* col_from = check_and_get_column<ColumnString>(column)) {
+ const typename ColumnString::Chars& data = col_from->get_chars();
+ const typename ColumnString::Offsets& offsets = col_from->get_offsets();
+ size_t size = offsets.size();
+ ColumnString::Offset current_offset = 0;
+ for (size_t i = 0; i < size; ++i) {
+ if constexpr (first) {
+ init_hash(col_to_data[i], reinterpret_cast<const char*>(&data[current_offset]),
+ offsets[i] - current_offset);
+ } else {
+ update_hash(col_to_data[i],
+ reinterpret_cast<const char*>(&data[current_offset]),
+ offsets[i] - current_offset);
+ }
+ current_offset = offsets[i];
+ }
+ } else if (const ColumnConst* col_from_const =
+ check_and_get_column_const_string_or_fixedstring(column)) {
+ auto value = col_from_const->get_value<TYPE_STRING>();
+ for (size_t i = 0; i < input_rows_count; ++i) {
+ if constexpr (first) {
+ init_hash(col_to_data[i], value.data(), value.size());
+ } else {
+ update_hash(col_to_data[i], value.data(), value.size());
+ }
+ }
+ } else {
+ DCHECK(false);
+ return Status::NotSupported("Illegal column {} of argument of function {}",
+ column->get_name(), name);
+ }
+ return Status::OK();
+ }
+
+private:
+ static __int128_t pack_hash(uint64_t h1, uint64_t h2) {
+ static_assert(sizeof(__int128_t) == sizeof(uint64_t) * 2);
+ // Store the two MurmurHash3 x64 128-bit lanes in a single LARGEINT value. Keep h1 in the
+ // low 64 bits and h2 in the high 64 bits to match murmur_hash3_x64_128's out[0]/out[1].
+ const auto value =
+ (static_cast<unsigned __int128>(h2) << 64) | static_cast<unsigned __int128>(h1);
+ return static_cast<__int128_t>(value);
+ }
+
+ static void unpack_hash(__int128_t value, uint64_t& h1, uint64_t& h2) {
+ static_assert(sizeof(__int128_t) == sizeof(uint64_t) * 2);
+ const auto unsigned_value = static_cast<unsigned __int128>(value);
+ h1 = static_cast<uint64_t>(unsigned_value);
+ h2 = static_cast<uint64_t>(unsigned_value >> 64);
+ }
+
+ static void init_hash(__int128_t& value, const void* data, size_t size) {
+ uint64_t hash[2] = {0, 0};
+ // The first SQL argument starts from seed 0, so it can use the existing 128-bit primitive
+ // directly. Later arguments must use update_hash() to continue from the saved (h1, h2).
+ murmur_hash3_x64_128(data, static_cast<int>(size), 0, hash);
Review Comment:
This narrows a SQL string length from `size_t` to `int` before entering `murmur_hash3_x64_128` / `murmur_hash3_x64_process`. Doris string offsets are not limited to `INT_MAX` (`StringLengthType` is `uint32_t`), so a large STRING value can wrap to a negative `len`; `murmur_hash3_x64_process` then computes a negative `nblocks` and derives `tail = data + nblocks * 16`, which can point before the buffer and read out of bounds. Please either make the 128-bit Murmur primitive accept a non-narrowing length type (as other hash wrappers do at their public boundary) or reject/guard values larger than `INT_MAX` before calling it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4448335291
# FE Regression Coverage Report
Increment line coverage ` 46.67% (14/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4447553971
<details>
<summary>TPC-DS: <b>Total hot run time: 170846 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c79471f90a154f510ebb6a6e7b3f8b314da95e4d, data reload: false
query5 4335 678 510 510
query6 327 219 203 203
query7 4264 544 311 311
query8 315 228 220 220
query9 8844 4019 3935 3935
query10 460 347 283 283
query11 5779 2371 2202 2202
query12 182 130 123 123
query13 1302 619 439 439
query14 6618 5349 5003 5003
query14_1 4287 4311 4306 4306
query15 210 202 181 181
query16 1047 441 423 423
query17 1106 736 618 618
query18 2727 488 344 344
query19 221 198 157 157
query20 137 133 126 126
query21 222 135 113 113
query22 13624 13576 13403 13403
query23 17174 16340 16171 16171
query23_1 16105 16195 16127 16127
query24 7465 1761 1356 1356
query24_1 1361 1348 1348 1348
query25 573 531 460 460
query26 1319 318 177 177
query27 2666 584 330 330
query28 4440 1956 1959 1956
query29 1042 652 546 546
query30 325 246 209 209
query31 1158 1072 954 954
query32 91 76 77 76
query33 553 373 312 312
query34 1170 1162 639 639
query35 784 788 686 686
query36 1321 1338 1165 1165
query37 151 99 104 99
query38 3206 3147 3065 3065
query39 939 942 907 907
query39_1 898 903 871 871
query40 257 155 138 138
query41 72 68 68 68
query42 113 109 110 109
query43 318 330 285 285
query44
query45 212 208 197 197
query46 1059 1204 728 728
query47 2276 2339 2237 2237
query48 404 447 295 295
query49 651 548 446 446
query50 701 299 227 227
query51 4311 4303 4194 4194
query52 107 108 98 98
query53 257 281 213 213
query54 325 285 274 274
query55 95 92 86 86
query56 332 317 324 317
query57 1437 1393 1290 1290
query58 322 279 281 279
query59 1539 1622 1451 1451
query60 354 345 339 339
query61 220 158 154 154
query62 673 620 550 550
query63 229 215 200 200
query64 2341 841 683 683
query65
query66 1682 510 401 401
query67 30175 29883 29771 29771
query68
query69 469 334 309 309
query70 1009 980 953 953
query71 312 262 265 262
query72 2949 2724 2455 2455
query73 813 768 428 428
query74 5063 4925 4752 4752
query75 2768 2664 2310 2310
query76 2265 1136 769 769
query77 408 444 342 342
query78 12893 12966 12324 12324
query79 1498 995 732 732
query80 869 586 463 463
query81 495 276 243 243
query82 1322 159 122 122
query83 355 281 248 248
query84 260 144 113 113
query85 912 533 445 445
query86 432 377 320 320
query87 3420 3372 3191 3191
query88 3559 2716 2664 2664
query89 456 387 331 331
query90 1778 181 179 179
query91 181 172 142 142
query92 78 77 76 76
query93 957 932 568 568
query94 626 337 319 319
query95 647 369 435 369
query96 1042 778 359 359
query97 2711 2685 2555 2555
query98 240 231 235 231
query99 1125 1103 984 984
Total cold run time: 254168 ms
Total hot run time: 170846 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4570023070
<details>
<summary>TPC-DS: <b>Total hot run time: 170924 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 76fb519e146749e3726ca0440c7c35d1094a27d8, data reload: false
query5 4384 651 537 537
query6 352 219 208 208
query7 4245 551 306 306
query8 345 228 227 227
query9 8871 4108 4043 4043
query10 445 345 288 288
query11 5835 2579 2211 2211
query12 183 138 129 129
query13 1303 591 444 444
query14 6144 5500 5168 5168
query14_1 4478 4496 4467 4467
query15 216 218 183 183
query16 3070 472 443 443
query17 1148 742 610 610
query18 2377 490 355 355
query19 230 207 164 164
query20 136 132 130 130
query21 220 141 114 114
query22 13641 13776 13471 13471
query23 17421 16428 16204 16204
query23_1 16390 16384 16257 16257
query24 7537 1785 1320 1320
query24_1 1299 1312 1323 1312
query25 618 504 444 444
query26 1268 327 182 182
query27 2784 555 349 349
query28 4443 2016 2013 2013
query29 1005 636 542 542
query30 329 242 199 199
query31 1116 1098 953 953
query32 98 72 72 72
query33 543 348 304 304
query34 1224 1133 655 655
query35 769 789 706 706
query36 1378 1411 1256 1256
query37 150 104 92 92
query38 3212 3204 3085 3085
query39 916 928 933 928
query39_1 867 897 880 880
query40 228 140 126 126
query41 64 64 63 63
query42 110 109 107 107
query43 325 331 298 298
query44
query45 218 209 194 194
query46 1098 1183 765 765
query47 2365 2352 2234 2234
query48 410 410 291 291
query49 624 487 395 395
query50 1000 354 256 256
query51 4399 4324 4318 4318
query52 106 105 91 91
query53 246 270 215 215
query54 309 277 260 260
query55 99 92 89 89
query56 309 306 292 292
query57 1429 1431 1349 1349
query58 299 270 267 267
query59 1687 1649 1412 1412
query60 319 334 303 303
query61 163 154 156 154
query62 708 645 590 590
query63 241 207 205 205
query64 2038 783 624 624
query65
query66 1591 468 358 358
query67 29803 29609 29468 29468
query68
query69 487 333 302 302
query70 1062 1002 966 966
query71 347 275 265 265
query72 2960 2680 2354 2354
query73 841 764 448 448
query74 5097 4921 4775 4775
query75 2674 2610 2270 2270
query76 1857 1136 786 786
query77 394 410 340 340
query78 12257 12440 11840 11840
query79 1495 1080 733 733
query80 1263 529 455 455
query81 505 274 241 241
query82 1366 155 122 122
query83 356 275 251 251
query84 267 144 116 116
query85 933 547 452 452
query86 443 353 334 334
query87 3427 3386 3247 3247
query88 3591 2704 2731 2704
query89 453 390 350 350
query90 1869 179 179 179
query91 179 168 140 140
query92 75 80 71 71
query93 1498 1398 854 854
query94 728 340 307 307
query95 679 385 444 385
query96 1015 791 343 343
query97 2719 2753 2590 2590
query98 238 230 228 228
query99 1173 1139 1012 1012
Total cold run time: 256429 ms
Total hot run time: 170924 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4574850559
<details>
<summary>TPC-DS: <b>Total hot run time: 172389 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3c305497fbefe4fad3e74d6a6cd5ea3a348873c1, data reload: false
query5 4311 662 502 502
query6 331 212 202 202
query7 4234 567 327 327
query8 338 240 233 233
query9 8828 4141 4074 4074
query10 462 349 313 313
query11 5784 2586 2237 2237
query12 186 127 135 127
query13 1305 619 457 457
query14 6084 5476 5174 5174
query14_1 4494 4495 4476 4476
query15 218 209 189 189
query16 1029 472 455 455
query17 1166 751 624 624
query18 2677 509 366 366
query19 218 211 172 172
query20 142 135 134 134
query21 220 145 122 122
query22 13784 13771 13349 13349
query23 17326 16622 16290 16290
query23_1 16406 16335 16288 16288
query24 7463 1782 1350 1350
query24_1 1347 1337 1325 1325
query25 598 501 454 454
query26 1321 321 177 177
query27 2665 557 351 351
query28 4392 2040 2007 2007
query29 1017 667 532 532
query30 315 242 202 202
query31 1128 1081 971 971
query32 90 84 78 78
query33 561 367 326 326
query34 1194 1117 660 660
query35 783 792 691 691
query36 1410 1408 1259 1259
query37 163 103 96 96
query38 3242 3181 3101 3101
query39 930 911 883 883
query39_1 890 866 874 866
query40 222 145 127 127
query41 65 64 66 64
query42 112 108 118 108
query43 333 333 296 296
query44
query45 210 206 195 195
query46 1113 1200 728 728
query47 2286 2356 2246 2246
query48 386 408 299 299
query49 643 504 380 380
query50 973 366 258 258
query51 4411 4307 4250 4250
query52 105 108 96 96
query53 255 284 225 225
query54 311 267 254 254
query55 98 91 86 86
query56 294 326 311 311
query57 1417 1404 1306 1306
query58 306 280 268 268
query59 1593 1695 1424 1424
query60 316 320 304 304
query61 162 155 152 152
query62 687 669 593 593
query63 242 203 227 203
query64 2400 795 652 652
query65
query66 1679 495 365 365
query67 29811 29831 29571 29571
query68
query69 463 350 306 306
query70 1067 995 957 957
query71 302 274 277 274
query72 3022 2753 2235 2235
query73 836 799 434 434
query74 5085 5012 4778 4778
query75 2698 2604 2266 2266
query76 2320 1188 775 775
query77 421 421 333 333
query78 12440 12385 11898 11898
query79 1493 1114 793 793
query80 670 532 461 461
query81 458 284 243 243
query82 1600 161 121 121
query83 376 303 257 257
query84 254 147 119 119
query85 872 543 457 457
query86 409 350 299 299
query87 3456 3377 3226 3226
query88 3620 2766 2740 2740
query89 442 388 341 341
query90 1979 191 181 181
query91 188 166 145 145
query92 89 80 74 74
query93 1438 1437 876 876
query94 577 355 337 337
query95 686 468 350 350
query96 1094 826 335 335
query97 2739 2758 2619 2619
query98 249 231 222 222
query99 1180 1172 1021 1021
Total cold run time: 255061 ms
Total hot run time: 172389 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4437379422
# FE UT Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4438056290
# BE Regression && UT Coverage Report
Increment line coverage `75.81% (47/62)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_e5a3a613d211ffe09d9dbcbdd50ecc8ef12a82a4_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.73% (27816/37727) |
| Line Coverage | 57.58% (301057/522826) |
| Region Coverage | 54.80% (251109/458234) |
| Branch Coverage | 56.29% (108578/192887) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4459491638
# FE UT Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4458972448
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4598150653
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4449357230
# FE UT Coverage Report
Increment line coverage ` 6.67% (1/15)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_c79471f90a154f510ebb6a6e7b3f8b314da95e4d/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #63196:
URL: https://github.com/apache/doris/pull/63196#discussion_r3257794460
##########
be/src/exprs/function/function_hash.cpp:
##########
@@ -133,6 +133,107 @@ using FunctionMurmurHash3_64_V2 =
using FunctionMurmurHash3U64V2 =
FunctionVariadicArgumentsBase<DataTypeInt128, MurmurHash3Impl<TYPE_LARGEINT, true>>;
+struct MurmurHash3128Impl {
+ static constexpr auto name = "murmur_hash3_128";
+
+ static Status empty_apply(IColumn& icolumn, size_t input_rows_count) {
Review Comment:
At least 1 arg? then should return error here
##########
regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy:
##########
@@ -100,6 +154,22 @@ suite("test_hash_function", "arrow_flight_sql") {
qt_mmh3_u64_v2_fold_1 "SELECT MURMUR_HASH3_U64_V2('test') + 1;"
qt_mmh3_u64_v2_fold_2 "SELECT MURMUR_HASH3_U64_V2('a', 'b') * 2;"
+ def validate_mmh3_128 = { String expected, String expression ->
Review Comment:
dont use this mode. directly use `qt_sql` and output the result to `.out` files
##########
regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy:
##########
@@ -100,6 +154,22 @@ suite("test_hash_function", "arrow_flight_sql") {
qt_mmh3_u64_v2_fold_1 "SELECT MURMUR_HASH3_U64_V2('test') + 1;"
qt_mmh3_u64_v2_fold_2 "SELECT MURMUR_HASH3_U64_V2('a', 'b') * 2;"
+ def validate_mmh3_128 = { String expected, String expression ->
+ def res = sql "SELECT MURMUR_HASH3_128(${expression});"
+ assertEquals(expected, res[0][0] == null ? null : res[0][0].toString());
+ }
+
+ validate_mmh3_128(null, "NULL");
Review Comment:
should also test for some more cases, like zero arg, other type inputs, many nulls and so on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4570696691
# FE Regression Coverage Report
Increment line coverage ` 9.16% (12/131)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_76fb519e146749e3726ca0440c7c35d1094a27d8_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4569857705
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](function) Support murmur_hash3_128 function [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63196:
URL: https://github.com/apache/doris/pull/63196#issuecomment-4577704387
# BE Regression && UT Coverage Report
Increment line coverage `95.00% (76/80)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63196_3c305497fbefe4fad3e74d6a6cd5ea3a348873c1_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.74% (28091/38097) |
| Line Coverage | 57.72% (305309/528982) |
| Region Coverage | 54.88% (255597/465727) |
| Branch Coverage | 56.38% (110397/195819) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org