You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "amorynan (via GitHub)" <gi...@apache.org> on 2024/06/17 11:45:23 UTC
[PR] [Feature](json_functions) support json-keys [doris]
amorynan opened a new pull request, #36411:
URL: https://github.com/apache/doris/pull/36411
## Proposed changes
support json_keys
sql reference MYSQL: https://dev.mysql.com/doc/refman/8.0/en/json-search-functions.html#function_json-keys
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2186207857
<details>
<summary>TPC-DS: <b>Total hot run time: 172499 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a92c05073799d6d858d02110e01c34da62d3b78c, data reload: false
query1 917 403 382 382
query2 6431 2391 2256 2256
query3 6645 209 211 209
query4 19036 17545 17432 17432
query5 4135 489 455 455
query6 247 166 170 166
query7 4599 324 294 294
query8 328 306 295 295
query9 8639 2490 2441 2441
query10 612 299 285 285
query11 10411 10076 9985 9985
query12 131 87 82 82
query13 1644 399 381 381
query14 9621 7493 6812 6812
query15 239 192 191 191
query16 7822 275 261 261
query17 1796 535 515 515
query18 1953 273 267 267
query19 194 151 154 151
query20 88 84 78 78
query21 210 132 128 128
query22 4396 4054 3966 3966
query23 33800 33114 33137 33114
query24 11872 2793 2824 2793
query25 652 358 357 357
query26 1793 156 155 155
query27 3020 314 326 314
query28 7268 2096 2071 2071
query29 1100 638 618 618
query30 288 150 149 149
query31 934 728 730 728
query32 97 53 59 53
query33 797 282 280 280
query34 978 461 454 454
query35 739 636 597 597
query36 1111 935 898 898
query37 192 75 76 75
query38 2885 2785 2713 2713
query39 840 813 780 780
query40 280 124 122 122
query41 55 53 58 53
query42 121 99 98 98
query43 543 567 536 536
query44 1241 716 739 716
query45 197 161 159 159
query46 1067 693 710 693
query47 1834 1772 1782 1772
query48 375 299 302 299
query49 1170 406 411 406
query50 776 392 413 392
query51 6933 6890 6718 6718
query52 143 88 97 88
query53 358 284 296 284
query54 1056 461 453 453
query55 75 77 73 73
query56 277 268 267 267
query57 1126 1033 1059 1033
query58 255 242 247 242
query59 3331 3313 3111 3111
query60 304 266 267 266
query61 91 90 89 89
query62 658 455 449 449
query63 319 295 283 283
query64 9889 2221 1719 1719
query65 3147 3105 3081 3081
query66 1372 324 324 324
query67 15506 14977 14957 14957
query68 4583 543 544 543
query69 452 300 305 300
query70 1081 1121 1136 1121
query71 391 269 271 269
query72 7196 6001 5407 5407
query73 751 333 328 328
query74 5902 5555 5456 5456
query75 3366 2645 2721 2645
query76 2512 961 923 923
query77 445 298 299 298
query78 10242 9888 9619 9619
query79 2253 510 516 510
query80 993 478 476 476
query81 588 222 221 221
query82 756 102 105 102
query83 240 167 167 167
query84 242 87 85 85
query85 1989 297 279 279
query86 504 327 311 311
query87 3268 3085 3142 3085
query88 3995 2441 2428 2428
query89 481 382 390 382
query90 1814 188 186 186
query91 126 107 100 100
query92 59 53 49 49
query93 2505 517 508 508
query94 1273 196 253 196
query95 413 320 314 314
query96 602 280 271 271
query97 3249 3027 3048 3027
query98 221 196 194 194
query99 1149 835 871 835
Total cold run time: 274600 ms
Total hot run time: 172499 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2186093537
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #36411:
URL: https://github.com/apache/doris/pull/36411#discussion_r1642691741
##########
be/src/vec/functions/function_jsonb.cpp:
##########
@@ -431,6 +431,176 @@ class FunctionJsonbExtract : public IFunction {
}
};
+class FunctionJsonbKeys : public IFunction {
+public:
+ static constexpr auto name = "json_keys";
+ static constexpr auto alias = "jsonb_keys";
+ static FunctionPtr create() { return std::make_shared<FunctionJsonbKeys>(); }
+ String get_name() const override { return name; }
+ bool is_variadic() const override { return true; }
+ size_t get_number_of_arguments() const override { return 0; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return make_nullable(
+ std::make_shared<DataTypeArray>(make_nullable(std::make_shared<DataTypeString>())));
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) const override {
+ DCHECK_GE(arguments.size(), 1);
+
+ ColumnPtr jsonb_data_column = nullptr;
+ const NullMap* data_null_map = nullptr;
+ // prepare jsonb data column
+ jsonb_data_column = unpack_if_const(block.get_by_position(arguments[0]).column).first;
+ if (block.get_by_position(arguments[0]).column->is_nullable()) {
+ const auto* nullable = check_and_get_column<ColumnNullable>(jsonb_data_column);
+ jsonb_data_column = nullable->get_nested_column_ptr();
+ data_null_map = &nullable->get_null_map_data();
+ }
+ const ColumnString* col_from_string = check_and_get_column<ColumnString>(jsonb_data_column);
+
+ // prepare parse path column prepare, maybe we do not have path column
+ ColumnPtr jsonb_path_column = nullptr;
+ const ColumnString* jsonb_path_col = nullptr;
+ bool path_const = false;
+ const NullMap* path_null_map = nullptr;
+ if (arguments.size() == 2) {
+ // we have should have a ColumnString for path
+ std::tie(jsonb_path_column, path_const) =
+ unpack_if_const(block.get_by_position(arguments[1]).column);
+ if (block.get_by_position(arguments[1]).column->is_nullable()) {
+ const auto* nullable = check_and_get_column<ColumnNullable>(jsonb_path_column);
+ jsonb_path_column = nullable->get_nested_column_ptr();
+ path_null_map = &nullable->get_null_map_data();
+ }
+ jsonb_path_col = check_and_get_column<ColumnString>(jsonb_path_column);
+ } else if (arguments.size() > 2) {
+ // here has argument param error
+ return Status::InvalidArgument("json_keys should have 1 or 2 arguments");
+ }
+
+ auto null_map = ColumnUInt8::create(input_rows_count, 0);
+ NullMap& res_null_map = null_map->get_data();
+
+ auto dst_arr = ColumnArray::create(
+ ColumnNullable::create(ColumnString::create(), ColumnUInt8::create()),
+ ColumnArray::ColumnOffsets::create());
+ ColumnNullable& dst_nested_column = assert_cast<ColumnNullable&>(dst_arr->get_data());
+
+ Status st;
+ if (jsonb_path_column) {
+ if (path_const) {
+ st = inner_loop_impl<true, true>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ } else {
+ st = inner_loop_impl<true, false>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ }
+ } else {
+ st = inner_loop_impl<false, false>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ }
+ if (!st.ok()) {
+ return st;
+ }
+ block.get_by_position(result).column =
+ ColumnNullable::create(std::move(dst_arr), std::move(null_map));
+ return st;
+ }
+
+private:
+ template <bool JSONB_PATH_PARAM, bool JSON_PATH_CONST>
+ static ALWAYS_INLINE Status inner_loop_impl(size_t input_rows_count, ColumnArray& dst_arr,
+ ColumnNullable& dst_nested_column,
+ NullMap& res_null_map,
+ const ColumnString& col_from_string,
+ const NullMap* jsonb_data_nullmap,
+ const ColumnString* jsonb_path_column,
+ const NullMap* path_null_map) {
+ // if path is const, we just need to parse it once
+ JsonbPath const_path;
+ if constexpr (JSONB_PATH_PARAM && JSON_PATH_CONST) {
+ StringRef r_raw_ref = jsonb_path_column->get_data_at(0);
+ if (!const_path.seek(r_raw_ref.data, r_raw_ref.size)) {
+ return Status::InvalidArgument(
+ "Json path error: {} for value: {}",
+ JsonbErrMsg::getErrMsg(JsonbErrType::E_INVALID_JSON_PATH),
+ r_raw_ref.to_string());
+ }
+ }
+ const auto& ldata = col_from_string.get_chars();
+ const auto& loffsets = col_from_string.get_offsets();
+ for (size_t i = 0; i < input_rows_count; ++i) {
+ // if jsonb data is null or path column is null , we should return null
+ if (jsonb_data_nullmap && (&jsonb_data_nullmap)[i]) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ if constexpr (JSONB_PATH_PARAM && !JSON_PATH_CONST) {
+ if (path_null_map && (&path_null_map)[i]) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ }
+ // extract jsonb keys
+ size_t l_off = loffsets[i - 1];
+ size_t l_size = loffsets[i] - l_off;
+ if (l_size == 0) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ const char* l_raw = reinterpret_cast<const char*>(&ldata[l_off]);
+ JsonbDocument* doc = JsonbDocument::createDocument(l_raw, l_size);
+ if (UNLIKELY(!doc || !doc->getValue())) {
Review Comment:
warning: boolean expression can be simplified by DeMorgan's theorem [readability-simplify-boolean-expr]
```cpp
if (UNLIKELY(!doc || !doc->getValue())) {
^
```
<details>
<summary>Additional context</summary>
**be/src/common/compiler_util.h:35:** expanded from macro 'UNLIKELY'
```cpp
#define UNLIKELY(expr) __builtin_expect(!!(expr), 0)
^
```
</details>
##########
be/src/vec/functions/function_jsonb.cpp:
##########
@@ -431,6 +431,176 @@
}
};
+class FunctionJsonbKeys : public IFunction {
+public:
+ static constexpr auto name = "json_keys";
+ static constexpr auto alias = "jsonb_keys";
+ static FunctionPtr create() { return std::make_shared<FunctionJsonbKeys>(); }
+ String get_name() const override { return name; }
+ bool is_variadic() const override { return true; }
+ size_t get_number_of_arguments() const override { return 0; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return make_nullable(
+ std::make_shared<DataTypeArray>(make_nullable(std::make_shared<DataTypeString>())));
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) const override {
+ DCHECK_GE(arguments.size(), 1);
+
+ ColumnPtr jsonb_data_column = nullptr;
+ const NullMap* data_null_map = nullptr;
+ // prepare jsonb data column
+ jsonb_data_column = unpack_if_const(block.get_by_position(arguments[0]).column).first;
+ if (block.get_by_position(arguments[0]).column->is_nullable()) {
+ const auto* nullable = check_and_get_column<ColumnNullable>(jsonb_data_column);
+ jsonb_data_column = nullable->get_nested_column_ptr();
+ data_null_map = &nullable->get_null_map_data();
+ }
+ const ColumnString* col_from_string = check_and_get_column<ColumnString>(jsonb_data_column);
+
+ // prepare parse path column prepare, maybe we do not have path column
+ ColumnPtr jsonb_path_column = nullptr;
+ const ColumnString* jsonb_path_col = nullptr;
+ bool path_const = false;
+ const NullMap* path_null_map = nullptr;
+ if (arguments.size() == 2) {
+ // we have should have a ColumnString for path
+ std::tie(jsonb_path_column, path_const) =
+ unpack_if_const(block.get_by_position(arguments[1]).column);
+ if (block.get_by_position(arguments[1]).column->is_nullable()) {
+ const auto* nullable = check_and_get_column<ColumnNullable>(jsonb_path_column);
+ jsonb_path_column = nullable->get_nested_column_ptr();
+ path_null_map = &nullable->get_null_map_data();
+ }
+ jsonb_path_col = check_and_get_column<ColumnString>(jsonb_path_column);
+ } else if (arguments.size() > 2) {
+ // here has argument param error
+ return Status::InvalidArgument("json_keys should have 1 or 2 arguments");
+ }
+
+ auto null_map = ColumnUInt8::create(input_rows_count, 0);
+ NullMap& res_null_map = null_map->get_data();
+
+ auto dst_arr = ColumnArray::create(
+ ColumnNullable::create(ColumnString::create(), ColumnUInt8::create()),
+ ColumnArray::ColumnOffsets::create());
+ ColumnNullable& dst_nested_column = assert_cast<ColumnNullable&>(dst_arr->get_data());
+
+ Status st;
+ if (jsonb_path_column) {
+ if (path_const) {
+ st = inner_loop_impl<true, true>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ } else {
+ st = inner_loop_impl<true, false>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ }
+ } else {
+ st = inner_loop_impl<false, false>(input_rows_count, *dst_arr, dst_nested_column,
+ res_null_map, *col_from_string, data_null_map,
+ jsonb_path_col, path_null_map);
+ }
+ if (!st.ok()) {
+ return st;
+ }
+ block.get_by_position(result).column =
+ ColumnNullable::create(std::move(dst_arr), std::move(null_map));
+ return st;
+ }
+
+private:
+ template <bool JSONB_PATH_PARAM, bool JSON_PATH_CONST>
+ static ALWAYS_INLINE Status inner_loop_impl(size_t input_rows_count, ColumnArray& dst_arr,
+ ColumnNullable& dst_nested_column,
+ NullMap& res_null_map,
+ const ColumnString& col_from_string,
+ const NullMap* jsonb_data_nullmap,
+ const ColumnString* jsonb_path_column,
+ const NullMap* path_null_map) {
+ // if path is const, we just need to parse it once
+ JsonbPath const_path;
+ if constexpr (JSONB_PATH_PARAM && JSON_PATH_CONST) {
+ StringRef r_raw_ref = jsonb_path_column->get_data_at(0);
+ if (!const_path.seek(r_raw_ref.data, r_raw_ref.size)) {
+ return Status::InvalidArgument(
+ "Json path error: {} for value: {}",
+ JsonbErrMsg::getErrMsg(JsonbErrType::E_INVALID_JSON_PATH),
+ r_raw_ref.to_string());
+ }
+ }
+ const auto& ldata = col_from_string.get_chars();
+ const auto& loffsets = col_from_string.get_offsets();
+ for (size_t i = 0; i < input_rows_count; ++i) {
+ // if jsonb data is null or path column is null , we should return null
+ if (jsonb_data_nullmap && (&jsonb_data_nullmap)[i]) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ if constexpr (JSONB_PATH_PARAM && !JSON_PATH_CONST) {
+ if (path_null_map && (&path_null_map)[i]) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ }
+ // extract jsonb keys
+ size_t l_off = loffsets[i - 1];
+ size_t l_size = loffsets[i] - l_off;
+ if (l_size == 0) {
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ const char* l_raw = reinterpret_cast<const char*>(&ldata[l_off]);
+ JsonbDocument* doc = JsonbDocument::createDocument(l_raw, l_size);
+ if (UNLIKELY(!doc || !doc->getValue())) {
+ dst_arr.clear();
+ return Status::InvalidArgument("jsonb data is invalid");
+ }
+ JsonbValue* obj_val;
+ if constexpr (JSONB_PATH_PARAM) {
+ if constexpr (!JSON_PATH_CONST) {
+ const ColumnString::Chars& rdata = jsonb_path_column->get_chars();
+ const ColumnString::Offsets& roffsets = jsonb_path_column->get_offsets();
+ size_t r_off = roffsets[i - 1];
+ size_t r_size = roffsets[i] - r_off;
+ const char* r_raw = reinterpret_cast<const char*>(&rdata[r_off]);
+ JsonbPath path;
+ if (!path.seek(r_raw, r_size)) {
+ return Status::InvalidArgument(
+ "Json path error: {} for value: {}",
+ JsonbErrMsg::getErrMsg(JsonbErrType::E_INVALID_JSON_PATH),
+ std::string_view(reinterpret_cast<const char*>(rdata.data()),
+ rdata.size()));
+ }
+ obj_val = doc->getValue()->findValue(path, nullptr);
+ } else {
+ obj_val = doc->getValue()->findValue(const_path, nullptr);
+ }
+ } else {
+ obj_val = doc->getValue();
+ }
+
+ if (!obj_val || !obj_val->isObject()) {
+ // if jsonb data is not object we should return null
+ res_null_map[i] = 1;
+ dst_arr.insert_default();
+ continue;
+ }
+ ObjectVal* obj = (ObjectVal*)obj_val;
Review Comment:
warning: use auto when initializing with a cast to avoid duplicating the type name [modernize-use-auto]
```suggestion
auto* obj = (ObjectVal*)obj_val;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2173171204
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2182014567
<details>
<summary>ClickBench: <b>Total hot run time: 30.83 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f15dd382b1a5e9a3c49505094c585edfbe350b31, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.08 0.07
query5 0.50 0.48 0.49
query6 1.15 0.72 0.73
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.54 0.50 0.48
query10 0.54 0.54 0.54
query11 0.15 0.12 0.12
query12 0.16 0.13 0.13
query13 0.59 0.59 0.59
query14 0.77 0.77 0.77
query15 0.86 0.81 0.82
query16 0.36 0.36 0.36
query17 0.97 0.96 1.01
query18 0.26 0.24 0.25
query19 1.80 1.78 1.73
query20 0.02 0.01 0.00
query21 15.42 0.67 0.67
query22 4.32 6.74 2.05
query23 18.38 1.35 1.33
query24 2.19 0.24 0.22
query25 0.16 0.09 0.08
query26 0.27 0.18 0.18
query27 0.08 0.08 0.08
query28 13.18 1.01 1.00
query29 12.65 3.36 3.39
query30 0.26 0.08 0.06
query31 2.86 0.41 0.39
query32 3.26 0.47 0.47
query33 2.82 2.96 2.85
query34 16.91 4.44 4.42
query35 4.46 4.44 4.48
query36 0.65 0.45 0.46
query37 0.19 0.15 0.16
query38 0.15 0.15 0.14
query39 0.04 0.04 0.04
query40 0.18 0.14 0.14
query41 0.10 0.05 0.05
query42 0.05 0.06 0.04
query43 0.05 0.05 0.04
Total cold run time: 109.38 s
Total hot run time: 30.83 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2186414275
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2186217673
<details>
<summary>ClickBench: <b>Total hot run time: 31.75 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a92c05073799d6d858d02110e01c34da62d3b78c, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.05 0.05
query4 1.67 0.08 0.07
query5 0.49 0.47 0.49
query6 1.14 0.72 0.71
query7 0.02 0.02 0.01
query8 0.05 0.04 0.05
query9 0.54 0.48 0.48
query10 0.54 0.55 0.54
query11 0.16 0.12 0.12
query12 0.15 0.12 0.13
query13 0.60 0.60 0.59
query14 0.77 0.77 0.83
query15 0.86 0.84 0.84
query16 0.38 0.37 0.37
query17 1.01 1.04 1.01
query18 0.22 0.23 0.26
query19 1.81 1.74 1.80
query20 0.01 0.01 0.01
query21 15.43 0.66 0.65
query22 3.80 6.84 2.76
query23 18.29 1.43 1.27
query24 2.09 0.23 0.22
query25 0.15 0.08 0.08
query26 0.26 0.17 0.18
query27 0.08 0.09 0.09
query28 13.26 1.07 1.03
query29 12.63 3.32 3.32
query30 0.26 0.06 0.05
query31 2.87 0.40 0.38
query32 3.24 0.48 0.48
query33 2.92 2.94 3.00
query34 17.00 4.58 4.43
query35 4.54 4.55 4.55
query36 0.67 0.49 0.48
query37 0.19 0.16 0.16
query38 0.15 0.16 0.15
query39 0.04 0.03 0.04
query40 0.17 0.14 0.14
query41 0.09 0.05 0.04
query42 0.05 0.05 0.04
query43 0.04 0.05 0.05
Total cold run time: 108.98 s
Total hot run time: 31.75 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2186185753
<details>
<summary>TPC-H: <b>Total hot run time: 39713 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a92c05073799d6d858d02110e01c34da62d3b78c, data reload: false
------ Round 1 ----------------------------------
q1 18075 4411 4425 4411
q2 2981 200 195 195
q3 11985 1096 1094 1094
q4 10881 808 849 808
q5 7958 2705 2593 2593
q6 216 131 130 130
q7 971 586 608 586
q8 9222 2063 2052 2052
q9 8603 6496 6448 6448
q10 8932 3679 3779 3679
q11 464 234 238 234
q12 435 224 224 224
q13 18804 2979 2976 2976
q14 263 228 220 220
q15 511 490 464 464
q16 532 392 374 374
q17 964 681 692 681
q18 8221 7424 7424 7424
q19 4809 1511 1337 1337
q20 655 315 333 315
q21 4884 3131 4019 3131
q22 394 341 337 337
Total cold run time: 120760 ms
Total hot run time: 39713 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4349 4206 4192 4192
q2 361 264 266 264
q3 3009 2782 2790 2782
q4 1864 1593 1586 1586
q5 5260 5269 5251 5251
q6 216 127 128 127
q7 2117 1757 1721 1721
q8 3187 3318 3307 3307
q9 8364 8286 8315 8286
q10 3877 3661 3627 3627
q11 584 474 506 474
q12 764 587 590 587
q13 16202 2987 2986 2986
q14 288 256 252 252
q15 528 479 485 479
q16 463 406 406 406
q17 1760 1474 1491 1474
q18 7619 7545 7308 7308
q19 1671 1588 1480 1480
q20 2065 1794 1797 1794
q21 4885 4620 4787 4620
q22 660 550 547 547
Total cold run time: 70093 ms
Total hot run time: 53550 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2174885216
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2174885257
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](json_functions) support json-keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36411:
URL: https://github.com/apache/doris/pull/36411#issuecomment-2182010294
<details>
<summary>TPC-DS: <b>Total hot run time: 170389 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f15dd382b1a5e9a3c49505094c585edfbe350b31, data reload: false
query1 945 383 385 383
query2 6462 2491 2323 2323
query3 6653 210 219 210
query4 19193 17406 17334 17334
query5 4188 465 481 465
query6 253 157 158 157
query7 4603 299 305 299
query8 334 296 299 296
query9 8465 2419 2400 2400
query10 601 295 291 291
query11 10589 10073 9964 9964
query12 137 84 92 84
query13 1629 361 358 358
query14 10227 7080 7591 7080
query15 236 192 184 184
query16 8002 261 275 261
query17 1903 568 520 520
query18 2040 275 277 275
query19 198 157 153 153
query20 93 86 89 86
query21 214 138 132 132
query22 4267 4061 3902 3902
query23 33754 33029 33088 33029
query24 11910 2891 2790 2790
query25 676 364 360 360
query26 1738 159 152 152
query27 2979 315 317 315
query28 7565 2083 2089 2083
query29 1048 635 608 608
query30 285 164 150 150
query31 956 739 758 739
query32 97 60 58 58
query33 769 298 278 278
query34 953 476 486 476
query35 731 608 603 603
query36 1094 921 890 890
query37 192 72 75 72
query38 2844 2740 2724 2724
query39 879 808 799 799
query40 276 129 125 125
query41 61 53 54 53
query42 123 96 104 96
query43 587 538 546 538
query44 1197 748 747 747
query45 200 170 169 169
query46 1096 731 712 712
query47 1857 1754 1747 1747
query48 368 305 298 298
query49 1204 445 407 407
query50 762 386 388 386
query51 6920 6851 6707 6707
query52 108 93 99 93
query53 362 290 294 290
query54 976 442 441 441
query55 77 73 78 73
query56 283 260 282 260
query57 1137 1082 1030 1030
query58 253 242 262 242
query59 3493 3383 3223 3223
query60 296 276 294 276
query61 96 94 91 91
query62 635 444 449 444
query63 350 304 295 295
query64 9827 2229 1737 1737
query65 3188 3268 3124 3124
query66 1389 349 332 332
query67 15233 14909 14967 14909
query68 4572 540 521 521
query69 448 309 333 309
query70 1193 1156 1163 1156
query71 385 281 286 281
query72 7230 5149 2990 2990
query73 744 325 323 323
query74 5850 5471 5452 5452
query75 3343 2677 2665 2665
query76 2456 997 1015 997
query77 430 303 293 293
query78 10619 9775 9708 9708
query79 2677 516 522 516
query80 2176 471 469 469
query81 581 217 215 215
query82 885 110 100 100
query83 275 177 174 174
query84 270 84 87 84
query85 2097 290 276 276
query86 496 309 333 309
query87 3232 3054 3059 3054
query88 4177 2373 2339 2339
query89 478 374 398 374
query90 1805 191 197 191
query91 128 116 100 100
query92 59 49 52 49
query93 2673 515 513 513
query94 1291 189 191 189
query95 412 312 317 312
query96 596 273 263 263
query97 3241 3026 3021 3021
query98 227 207 196 196
query99 1213 838 822 822
Total cold run time: 278315 ms
Total hot run time: 170389 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org