You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "taptao (via GitHub)" <gi...@apache.org> on 2024/03/06 00:38:53 UTC
[PR] [Improve](InPredict) enhance in predict with array type [doris]
taptao opened a new pull request, #31828:
URL: https://github.com/apache/doris/pull/31828
## Proposed changes
Issue Number: close https://github.com/apache/doris/issues/31157
Support in predict with array type.
Example:select [1,2] in ([1,2],[3,4])
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987205368
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 75aee4ad0f3ec4a743d6ed86b68c6d93d517876a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 17.5 seconds inserted 10000000 Rows, about 571K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987708681
TeamCity be ut coverage result:
Function Coverage: 35.00% (8556/24446)
Line Coverage: 26.83% (69533/259141)
Region Coverage: 26.08% (36073/138337)
Branch Coverage: 23.05% (18444/80032)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e009df429843d6f14dee1c9d644fe85a4a69b117_e009df429843d6f14dee1c9d644fe85a4a69b117/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1518706256
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
+#include <stddef.h>
+
+#include <algorithm>
+#include <memory>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+#include "common/status.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_struct.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_struct_in.h"
+
+namespace doris {
+namespace vectorized {
+struct ColumnRowRef;
+}
+} // namespace doris
+
+namespace doris::vectorized {
+
+struct ArrayInState {
Review Comment:
code is duplicated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1519135511
##########
be/src/vec/functions/function_collection_in.h:
##########
@@ -115,36 +118,35 @@ class FunctionStructIn : public IFunction {
const auto& [col, _] = unpack_if_const(const_column_ptr->column_ptr);
if (col->is_nullable()) {
auto* null_col = vectorized::check_and_get_column<vectorized::ColumnNullable>(col);
- if (null_col->has_null()) {
+ if (!state->null_in_set && null_col->has_null()) {
Review Comment:
this if else logic maybe not right !
think about this in params: (null, xxx, null), the second null will be inserted into args_column_ptr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987670356
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987192026
TeamCity be ut coverage result:
Function Coverage: 35.36% (8555/24196)
Line Coverage: 27.15% (69533/256138)
Region Coverage: 26.37% (36082/136836)
Branch Coverage: 23.22% (18443/79444)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f773e040b642075a1d78708083ad3f2a8e0721cb_f773e040b642075a1d78708083ad3f2a8e0721cb/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987247064
<details>
<summary>ClickBench: <b>Total hot run time: 30.87 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1624e1fd50f97670e5d5977a92c3b0eebd1dd69a, data reload: false
query1 0.04 0.03 0.04
query2 0.06 0.02 0.02
query3 0.24 0.06 0.06
query4 1.67 0.10 0.10
query5 0.53 0.50 0.50
query6 1.30 0.69 0.68
query7 0.01 0.01 0.01
query8 0.04 0.03 0.03
query9 0.57 0.51 0.53
query10 0.58 0.56 0.57
query11 0.14 0.10 0.10
query12 0.13 0.11 0.10
query13 0.58 0.57 0.57
query14 0.75 0.77 0.74
query15 0.82 0.80 0.79
query16 0.38 0.37 0.36
query17 1.03 0.98 0.98
query18 0.27 0.25 0.26
query19 1.78 1.73 1.73
query20 0.02 0.01 0.01
query21 15.40 0.67 0.61
query22 3.43 4.05 2.29
query23 17.72 1.11 1.02
query24 2.12 0.59 0.38
query25 0.29 0.11 0.05
query26 0.16 0.14 0.14
query27 0.03 0.03 0.02
query28 12.10 0.83 0.84
query29 12.58 3.53 3.18
query30 0.58 0.58 0.58
query31 2.78 0.35 0.35
query32 3.38 0.43 0.43
query33 2.95 2.90 2.91
query34 15.48 4.33 4.32
query35 4.32 4.34 4.34
query36 1.08 1.03 1.02
query37 0.07 0.06 0.06
query38 0.05 0.03 0.03
query39 0.02 0.01 0.02
query40 0.17 0.12 0.13
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.03 0.02
Total cold run time: 105.78 s
Total hot run time: 30.87 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1519136953
##########
be/src/vec/functions/function_collection_in.h:
##########
@@ -155,18 +157,25 @@ class FunctionStructIn : public IFunction {
const ColumnWithTypeAndName& left_arg = block.get_by_position(arguments[0]);
const auto& [materialized_column, col_const] = unpack_if_const(left_arg.column);
+ auto materialized_column_not_null = materialized_column;
+ if (materialized_column_not_null->is_nullable()) {
+ materialized_column_not_null = assert_cast<ColumnPtr>(
+ vectorized::check_and_get_column<vectorized::ColumnNullable>(
+ materialized_column_not_null)
+ ->get_nested_column_ptr());
+ }
for (size_t i = 0; i < input_rows_count; ++i) {
- bool find = args_set.find({materialized_column, i}) != args_set.end();
+ bool find = args_set.find({materialized_column_not_null, i}) != args_set.end();
+
if constexpr (negative) {
vec_res[i] = !find;
} else {
vec_res[i] = find;
}
- if (in_state->null_in_set) {
- vec_null_map_to[i] = negative == vec_res[i];
- } else {
- vec_null_map_to[i] = false;
+
+ if (!find && null_in_set) {
Review Comment:
do not make if not have else as possible as u can ! make code clear for read . and as less as condition in if , the compiler can make more optimize behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1988222016
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1518583661
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
Review Comment:
warning: 'glog/logging.h' file not found [clang-diagnostic-error]
```cpp
#include <glog/logging.h>
^
```
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
+#include <stddef.h>
+
+#include <algorithm>
+#include <memory>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+#include "common/status.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_struct.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_struct_in.h"
+
+namespace doris {
+namespace vectorized {
+struct ColumnRowRef;
+}
+} // namespace doris
+
+namespace doris::vectorized {
+
+struct ArrayInState {
+ ENABLE_FACTORY_CREATOR(ArrayInState)
+ std::unordered_set<ColumnRowRef, ColumnRowRef> args_set;
+ bool null_in_set = false;
+};
+
+template <bool negative>
+class FunctionArrayIn : public IFunction {
+public:
+ static constexpr auto name = negative ? "array_not_in" : "array_in";
+
+ static FunctionPtr create() { return std::make_shared<FunctionArrayIn>(); }
+
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return true; }
+
+ size_t get_number_of_arguments() const override { return 0; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& args) const override {
+ for (const auto& arg : args) {
+ if (arg->is_nullable()) {
+ return make_nullable(std::make_shared<DataTypeUInt8>());
+ }
+ }
+ return std::make_shared<DataTypeUInt8>();
+ }
+
+ bool use_default_implementation_for_nulls() const override { return false; }
+
+ // make data in context into a set
+ Status open(FunctionContext* context, FunctionContext::FunctionStateScope scope) override {
+ if (scope == FunctionContext::THREAD_LOCAL) {
+ return Status::OK();
+ }
+
+ int num_args = context->get_num_args();
+ DCHECK(num_args >= 1);
+
+ const TypeDescriptor* arg_type_desc = context->get_arg_type(0);
+ DataTypePtr data_type_ptr =
+ DataTypeFactory::instance().create_data_type(*arg_type_desc, false);
+ MutableColumnPtr args_column_ptr = remove_nullable(data_type_ptr)->create_column();
+
+ std::shared_ptr<ArrayInState> array_in_state = std::make_shared<ArrayInState>();
+ context->set_function_state(FunctionContext::FRAGMENT_LOCAL, array_in_state);
+
+ // put const column to args_column
+ for (int i = 1; i < num_args; i++) {
+ ColumnPtrWrapper* const_col_ptr = context->get_constant_col(i);
+ if (const_col_ptr == nullptr) {
+ break;
+ }
+ const auto& [col, _] = unpack_if_const(const_col_ptr->column_ptr);
+ if (col->is_nullable()) {
+ auto* null_col = vectorized::check_and_get_column<vectorized::ColumnNullable>(col);
+ if (!array_in_state->null_in_set && null_col->has_null()) {
+ array_in_state->null_in_set = true;
+ } else {
+ args_column_ptr->insert_from(null_col->get_nested_column(), 0);
+ }
+ } else {
+ args_column_ptr->insert_from(*col, 0);
+ }
+ }
+ ColumnPtr column_ptr = std::move(args_column_ptr);
+ int col_size = column_ptr->size();
+ for (size_t i = 0; i < col_size; i++) {
+ array_in_state->args_set.insert({column_ptr, i});
+ }
+
+ return Status::OK();
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) const override {
+ auto in_state = reinterpret_cast<ArrayInState*>(
Review Comment:
warning: 'auto in_state' can be declared as 'auto *in_state' [readability-qualified-auto]
```suggestion
auto *in_state = reinterpret_cast<ArrayInState*>(
```
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
+#include <stddef.h>
+
+#include <algorithm>
+#include <memory>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+#include "common/status.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_struct.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_struct_in.h"
+
+namespace doris {
+namespace vectorized {
Review Comment:
warning: nested namespaces can be concatenated [modernize-concat-nested-namespaces]
```suggestion
namespace doris::vectorized {
```
be/src/vec/functions/function_array_in.h:46:
```diff
- }
- } // namespace doris
+ } // namespace doris
```
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
+#include <stddef.h>
Review Comment:
warning: inclusion of deprecated C++ header 'stddef.h'; consider using 'cstddef' instead [modernize-deprecated-headers]
```suggestion
#include <cstddef>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987198909
<details>
<summary>TPC-H: <b>Total hot run time: 38085 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 75aee4ad0f3ec4a743d6ed86b68c6d93d517876a, data reload: false
------ Round 1 ----------------------------------
q1 17585 4126 4011 4011
q2 2030 148 142 142
q3 10668 945 941 941
q4 4703 956 956 956
q5 7589 2871 3021 2871
q6 180 121 129 121
q7 1282 831 818 818
q8 9593 2072 2078 2072
q9 7271 6505 6457 6457
q10 8235 2586 2562 2562
q11 423 218 221 218
q12 739 327 324 324
q13 17971 2933 2949 2933
q14 291 262 254 254
q15 498 441 441 441
q16 496 404 399 399
q17 957 907 890 890
q18 6815 5971 5894 5894
q19 1569 1523 1519 1519
q20 542 297 300 297
q21 7546 3691 3673 3673
q22 853 292 297 292
Total cold run time: 107836 ms
Total hot run time: 38085 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4008 3964 4005 3964
q2 333 235 227 227
q3 2990 2918 2969 2918
q4 1847 1842 1810 1810
q5 5241 5292 5246 5246
q6 212 115 115 115
q7 2264 1838 1861 1838
q8 3223 3306 3291 3291
q9 8558 8584 8546 8546
q10 6165 3734 3690 3690
q11 527 460 446 446
q12 704 559 546 546
q13 6804 2863 2829 2829
q14 280 254 259 254
q15 485 433 438 433
q16 456 428 421 421
q17 1689 1668 1665 1665
q18 7915 7426 7310 7310
q19 1630 1609 1615 1609
q20 1912 1732 1748 1732
q21 5067 4821 4771 4771
q22 570 478 474 474
Total cold run time: 62880 ms
Total hot run time: 54135 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987180370
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987710199
<details>
<summary>TPC-DS: <b>Total hot run time: 186213 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e009df429843d6f14dee1c9d644fe85a4a69b117, data reload: false
query1 927 356 341 341
query2 7381 1998 1953 1953
query3 6704 217 219 217
query4 24918 20994 20946 20946
query5 4315 1441 502 502
query6 269 178 202 178
query7 4627 320 312 312
query8 243 189 171 171
query9 8437 2185 2182 2182
query10 443 240 248 240
query11 15025 14553 14444 14444
query12 136 89 87 87
query13 1624 442 418 418
query14 10195 10016 9620 9620
query15 245 201 196 196
query16 7784 276 278 276
query17 1697 631 572 572
query18 1976 292 293 292
query19 206 169 171 169
query20 117 87 96 87
query21 200 130 125 125
query22 4576 4427 4419 4419
query23 31535 31014 30924 30924
query24 11527 3039 3042 3039
query25 695 399 384 384
query26 1542 169 169 169
query27 3084 368 356 356
query28 6708 1812 1806 1806
query29 1047 610 598 598
query30 301 152 154 152
query31 927 739 728 728
query32 112 65 62 62
query33 742 272 265 265
query34 970 474 471 471
query35 862 647 641 641
query36 913 809 814 809
query37 158 79 77 77
query38 3303 3161 3133 3133
query39 1436 1368 1388 1368
query40 230 126 143 126
query41 68 62 62 62
query42 114 102 106 102
query43 437 398 416 398
query44 1099 721 716 716
query45 273 252 236 236
query46 1049 801 786 786
query47 1626 1572 1561 1561
query48 431 345 340 340
query49 1155 341 365 341
query50 779 380 383 380
query51 6689 6506 6683 6506
query52 108 98 99 98
query53 365 296 285 285
query54 349 264 254 254
query55 82 86 84 84
query56 257 227 230 227
query57 1084 999 997 997
query58 256 223 238 223
query59 2614 2441 2404 2404
query60 277 292 280 280
query61 116 112 112 112
query62 665 419 417 417
query63 313 286 295 286
query64 6612 3453 3130 3130
query65 3077 3030 3010 3010
query66 1383 325 316 316
query67 14957 14274 14777 14274
query68 9301 573 581 573
query69 661 404 397 397
query70 1354 1104 1130 1104
query71 520 301 284 284
query72 6526 2656 2531 2531
query73 1567 337 316 316
query74 6884 6563 6492 6492
query75 9595 8035 8018 8018
query76 5604 1068 1160 1068
query77 645 260 250 250
query78 10271 9534 9585 9534
query79 10846 522 528 522
query80 1390 402 445 402
query81 481 210 215 210
query82 398 195 208 195
query83 220 151 149 149
query84 284 78 83 78
query85 1114 342 332 332
query86 390 300 303 300
query87 3436 3222 3250 3222
query88 4216 2294 2282 2282
query89 539 369 368 368
query90 2024 180 187 180
query91 164 130 125 125
query92 59 47 49 47
query93 6264 534 523 523
query94 1329 189 188 188
query95 452 349 352 349
query96 613 270 261 261
query97 4406 4257 4282 4257
query98 234 216 212 212
query99 1151 770 764 764
Total cold run time: 307310 ms
Total hot run time: 186213 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987202398
<details>
<summary>TPC-DS: <b>Total hot run time: 187069 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 75aee4ad0f3ec4a743d6ed86b68c6d93d517876a, data reload: false
query1 936 358 339 339
query2 7389 2104 2023 2023
query3 6706 212 211 211
query4 26495 21030 21062 21030
query5 4292 457 481 457
query6 270 180 178 178
query7 4621 321 317 317
query8 233 180 178 178
query9 8323 2291 2299 2291
query10 473 239 244 239
query11 14965 14563 14201 14201
query12 135 92 89 89
query13 1659 434 444 434
query14 10597 9698 9829 9698
query15 246 204 201 201
query16 7956 286 278 278
query17 1513 589 571 571
query18 1971 291 295 291
query19 201 174 166 166
query20 101 90 92 90
query21 201 140 133 133
query22 4658 4457 4413 4413
query23 31583 30957 30917 30917
query24 11662 3122 3084 3084
query25 678 413 394 394
query26 1661 173 165 165
query27 3049 372 372 372
query28 6389 1889 1866 1866
query29 1089 607 603 603
query30 308 150 147 147
query31 922 748 743 743
query32 106 69 68 68
query33 776 261 257 257
query34 1038 482 489 482
query35 851 637 643 637
query36 926 815 842 815
query37 142 74 80 74
query38 3220 3117 3133 3117
query39 1412 1386 1383 1383
query40 301 118 114 114
query41 63 55 56 55
query42 111 105 103 103
query43 439 396 403 396
query44 1099 721 729 721
query45 283 257 254 254
query46 1054 807 778 778
query47 1648 1553 1551 1551
query48 427 357 357 357
query49 1236 364 343 343
query50 784 381 383 381
query51 6852 6702 6652 6652
query52 103 99 97 97
query53 350 286 292 286
query54 308 273 263 263
query55 88 86 81 81
query56 260 222 234 222
query57 1079 1002 979 979
query58 248 227 229 227
query59 2502 2281 2413 2281
query60 267 254 261 254
query61 135 117 117 117
query62 643 422 413 413
query63 309 285 280 280
query64 5951 3209 3144 3144
query65 3051 3009 3007 3007
query66 1088 334 317 317
query67 15131 14690 14517 14517
query68 16309 548 584 548
query69 748 389 379 379
query70 1937 1065 1087 1065
query71 593 272 267 267
query72 10273 2619 2479 2479
query73 3974 338 345 338
query74 7248 6905 6979 6905
query75 12378 8013 8066 8013
query76 7351 1113 1193 1113
query77 1177 258 250 250
query78 10206 9758 9608 9608
query79 8963 503 510 503
query80 938 384 385 384
query81 489 215 207 207
query82 332 195 196 195
query83 413 139 135 135
query84 286 84 78 78
query85 1226 337 331 331
query86 372 307 296 296
query87 3437 3216 3197 3197
query88 3175 2376 2351 2351
query89 527 352 362 352
query90 2288 171 170 170
query91 165 130 134 130
query92 63 50 51 50
query93 2840 537 533 533
query94 1499 192 189 189
query95 459 345 338 338
query96 588 267 263 263
query97 4429 4279 4311 4279
query98 223 220 221 220
query99 1054 790 761 761
Total cold run time: 321523 ms
Total hot run time: 187069 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987194166
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1519034861
##########
be/src/vec/functions/function_array_in.h:
##########
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+
+#pragma once
+
+#include <glog/logging.h>
+#include <stddef.h>
+
+#include <algorithm>
+#include <memory>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+#include "common/status.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/column_struct.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/block.h"
+#include "vec/data_types/data_type_factory.hpp"
+#include "vec/data_types/data_type_nullable.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_struct_in.h"
+
+namespace doris {
+namespace vectorized {
+struct ColumnRowRef;
+}
+} // namespace doris
+
+namespace doris::vectorized {
+
+struct ArrayInState {
Review Comment:
done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1986879502
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987044630
just keep a function_collection_in.h/cpp file to handle array/struct/map this collection type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987239281
TeamCity be ut coverage result:
Function Coverage: 35.36% (8555/24196)
Line Coverage: 27.14% (69523/256138)
Region Coverage: 26.37% (36078/136836)
Branch Coverage: 23.22% (18443/79444)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1624e1fd50f97670e5d5977a92c3b0eebd1dd69a_1624e1fd50f97670e5d5977a92c3b0eebd1dd69a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "taptao (via GitHub)" <gi...@apache.org>.
taptao commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987227370
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987202787
TeamCity be ut coverage result:
Function Coverage: 35.36% (8555/24196)
Line Coverage: 27.14% (69517/256138)
Region Coverage: 26.36% (36066/136836)
Branch Coverage: 23.21% (18439/79444)
Coverage Report: http://coverage.selectdb-in.cc/coverage/75aee4ad0f3ec4a743d6ed86b68c6d93d517876a_75aee4ad0f3ec4a743d6ed86b68c6d93d517876a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #31828:
URL: https://github.com/apache/doris/pull/31828#discussion_r1519131053
##########
be/src/vec/functions/function_collection_in.h:
##########
@@ -98,14 +98,17 @@ class FunctionStructIn : public IFunction {
if (scope == FunctionContext::THREAD_LOCAL) {
return Status::OK();
}
- std::shared_ptr<StructInState> state = std::make_shared<StructInState>();
+ int num_args = context->get_num_args();
+ DCHECK(num_args >= 1);
+
+ std::shared_ptr<CollectionInState> state = std::make_shared<CollectionInState>();
context->set_function_state(scope, state);
- DCHECK(context->get_num_args() >= 1);
+
auto* col_desc = context->get_arg_type(0);
- DataTypePtr args_type = DataTypeFactory::instance().create_data_type(*col_desc);
- MutableColumnPtr column_struct_ptr_args = remove_nullable(args_type)->create_column();
- NullMap null_map(context->get_num_args(), false);
- for (int i = 1; i < context->get_num_args(); ++i) {
+ DataTypePtr args_type = DataTypeFactory::instance().create_data_type(*col_desc, false);
+ MutableColumnPtr args_column_ptr = remove_nullable(args_type)->create_column();
Review Comment:
create_data_type(*col_desc, false); u set false already , there is no need to remove_nullable again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1988093525
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #31828:
URL: https://github.com/apache/doris/pull/31828
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1986892972
TeamCity be ut coverage result:
Function Coverage: 35.35% (8556/24205)
Line Coverage: 27.13% (69522/256232)
Region Coverage: 26.35% (36073/136885)
Branch Coverage: 23.20% (18435/79470)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d9f8c5446d7dc66b23313b44a7ea4891483bdb95_d9f8c5446d7dc66b23313b44a7ea4891483bdb95/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987698954
<details>
<summary>TPC-H: <b>Total hot run time: 38520 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e009df429843d6f14dee1c9d644fe85a4a69b117, data reload: false
------ Round 1 ----------------------------------
q1 18541 4211 4180 4180
q2 2751 159 150 150
q3 11305 975 971 971
q4 5242 971 974 971
q5 8161 2954 3084 2954
q6 181 125 129 125
q7 1349 867 873 867
q8 9993 2073 2065 2065
q9 7300 6499 6483 6483
q10 8216 2583 2561 2561
q11 425 229 224 224
q12 770 325 317 317
q13 17951 3014 3014 3014
q14 277 253 249 249
q15 508 482 459 459
q16 487 404 408 404
q17 942 870 851 851
q18 6625 5900 5914 5900
q19 1570 1531 1512 1512
q20 551 286 286 286
q21 7536 3677 3724 3677
q22 785 300 301 300
Total cold run time: 111466 ms
Total hot run time: 38520 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4010 4032 4018 4018
q2 321 231 228 228
q3 2959 2886 2954 2886
q4 1848 1838 1881 1838
q5 5276 5292 5277 5277
q6 205 116 120 116
q7 2283 1865 1828 1828
q8 3216 3303 3295 3295
q9 8538 8558 8563 8558
q10 6141 3721 3720 3720
q11 548 440 440 440
q12 701 563 555 555
q13 3643 2891 2839 2839
q14 273 257 260 257
q15 498 462 448 448
q16 457 430 405 405
q17 1687 1674 1664 1664
q18 7961 7439 7324 7324
q19 1655 1624 1625 1624
q20 1913 1693 1713 1693
q21 4998 4774 4803 4774
q22 540 447 442 442
Total cold run time: 59671 ms
Total hot run time: 54229 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987247911
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1624e1fd50f97670e5d5977a92c3b0eebd1dd69a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 16.0 seconds inserted 10000000 Rows, about 625K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987204427
<details>
<summary>ClickBench: <b>Total hot run time: 30.59 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 75aee4ad0f3ec4a743d6ed86b68c6d93d517876a, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.23 0.06 0.06
query4 1.67 0.09 0.10
query5 0.50 0.50 0.51
query6 1.24 0.68 0.68
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.56 0.52 0.51
query10 0.58 0.58 0.57
query11 0.13 0.10 0.10
query12 0.12 0.10 0.10
query13 0.58 0.57 0.57
query14 0.73 0.75 0.75
query15 0.82 0.80 0.81
query16 0.38 0.40 0.39
query17 0.97 1.00 0.97
query18 0.25 0.25 0.26
query19 1.82 1.74 1.68
query20 0.01 0.02 0.01
query21 15.43 0.63 0.60
query22 3.24 3.65 2.28
query23 17.30 1.02 0.96
query24 2.12 0.38 0.19
query25 0.27 0.12 0.05
query26 0.17 0.14 0.14
query27 0.03 0.04 0.03
query28 12.53 0.84 0.84
query29 12.56 3.23 3.29
query30 0.59 0.58 0.58
query31 2.78 0.35 0.34
query32 3.39 0.44 0.44
query33 2.98 2.90 2.90
query34 15.49 4.31 4.29
query35 4.31 4.32 4.28
query36 1.09 1.01 1.04
query37 0.07 0.05 0.06
query38 0.04 0.04 0.03
query39 0.02 0.02 0.02
query40 0.18 0.14 0.14
query41 0.07 0.02 0.02
query42 0.02 0.02 0.02
query43 0.02 0.03 0.03
Total cold run time: 105.45 s
Total hot run time: 30.59 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987245037
<details>
<summary>TPC-DS: <b>Total hot run time: 187160 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1624e1fd50f97670e5d5977a92c3b0eebd1dd69a, data reload: false
query1 910 342 337 337
query2 7374 2153 2072 2072
query3 6704 217 207 207
query4 26687 21146 21152 21146
query5 4315 471 472 471
query6 279 183 179 179
query7 4627 311 302 302
query8 233 174 168 168
query9 8394 2296 2291 2291
query10 458 245 250 245
query11 14956 14639 14288 14288
query12 135 90 87 87
query13 1660 451 437 437
query14 11164 10175 9849 9849
query15 267 200 200 200
query16 8111 287 284 284
query17 1709 581 552 552
query18 2102 280 266 266
query19 204 160 161 160
query20 101 92 89 89
query21 203 129 126 126
query22 4596 4432 4417 4417
query23 30953 30995 30514 30514
query24 10659 3083 3011 3011
query25 643 371 374 371
query26 1501 165 166 165
query27 3035 378 362 362
query28 6505 1863 1861 1861
query29 1001 621 613 613
query30 310 161 150 150
query31 928 736 762 736
query32 107 64 62 62
query33 766 274 277 274
query34 984 474 498 474
query35 851 641 630 630
query36 933 841 847 841
query37 151 86 89 86
query38 3164 3159 3128 3128
query39 1431 1388 1387 1387
query40 220 132 125 125
query41 61 55 54 54
query42 113 105 104 104
query43 430 398 402 398
query44 1107 732 716 716
query45 296 250 258 250
query46 1071 836 784 784
query47 1627 1568 1537 1537
query48 439 351 358 351
query49 1167 357 352 352
query50 794 396 400 396
query51 6736 6706 6707 6706
query52 103 95 94 94
query53 363 293 286 286
query54 297 259 279 259
query55 86 85 85 85
query56 248 238 229 229
query57 1072 1002 993 993
query58 248 231 236 231
query59 2500 2405 2394 2394
query60 311 288 257 257
query61 135 117 116 116
query62 663 407 407 407
query63 309 283 283 283
query64 5055 3465 3568 3465
query65 3041 3026 3034 3026
query66 944 370 351 351
query67 15048 14505 14267 14267
query68 12377 578 589 578
query69 761 406 394 394
query70 1376 1158 1120 1120
query71 630 294 297 294
query72 10224 2634 2451 2451
query73 1996 341 346 341
query74 7247 6841 6739 6739
query75 11878 7916 7969 7916
query76 8104 1100 1188 1100
query77 989 273 271 271
query78 10214 9568 9489 9489
query79 10471 542 545 542
query80 1170 404 462 404
query81 486 216 205 205
query82 333 202 200 200
query83 240 148 143 143
query84 279 81 78 78
query85 1234 340 329 329
query86 365 316 276 276
query87 3418 3184 3186 3184
query88 3474 2375 2380 2375
query89 539 365 363 363
query90 2354 180 180 180
query91 164 127 128 127
query92 59 53 51 51
query93 4742 545 545 545
query94 1610 205 193 193
query95 464 359 355 355
query96 607 268 267 267
query97 4421 4327 4312 4312
query98 224 233 215 215
query99 1083 777 769 769
Total cold run time: 317128 ms
Total hot run time: 187160 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987241156
<details>
<summary>TPC-H: <b>Total hot run time: 37535 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1624e1fd50f97670e5d5977a92c3b0eebd1dd69a, data reload: false
------ Round 1 ----------------------------------
q1 17641 3995 3977 3977
q2 2016 142 138 138
q3 10568 966 923 923
q4 4656 919 934 919
q5 7587 2850 2973 2850
q6 176 126 125 125
q7 1267 822 810 810
q8 9482 1976 2045 1976
q9 7248 6417 6385 6385
q10 8211 2558 2550 2550
q11 415 223 215 215
q12 796 326 317 317
q13 17985 2918 2908 2908
q14 275 248 251 248
q15 487 447 440 440
q16 489 388 392 388
q17 938 863 825 825
q18 6609 5878 5794 5794
q19 1548 1511 1510 1510
q20 533 289 280 280
q21 7547 3674 3670 3670
q22 788 292 287 287
Total cold run time: 107262 ms
Total hot run time: 37535 ms
----- Round 2, with runtime_filter_mode=off -----
q1 3991 3950 3975 3950
q2 320 229 222 222
q3 2947 2905 2881 2881
q4 1867 1772 1765 1765
q5 5234 5209 5194 5194
q6 200 116 118 116
q7 2256 1814 1820 1814
q8 3174 3297 3248 3248
q9 8505 8466 8522 8466
q10 6134 3725 3692 3692
q11 530 450 433 433
q12 696 564 531 531
q13 15050 2765 2784 2765
q14 291 264 252 252
q15 476 441 437 437
q16 453 425 423 423
q17 1674 1649 1655 1649
q18 7875 7432 7314 7314
q19 1801 1598 1607 1598
q20 1928 1786 1690 1690
q21 4965 4745 4840 4745
q22 544 471 470 470
Total cold run time: 70911 ms
Total hot run time: 53655 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987715550
<details>
<summary>ClickBench: <b>Total hot run time: 29.78 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e009df429843d6f14dee1c9d644fe85a4a69b117, data reload: false
query1 0.04 0.03 0.04
query2 0.06 0.03 0.02
query3 0.24 0.06 0.06
query4 1.66 0.09 0.10
query5 0.50 0.49 0.50
query6 1.21 0.68 0.67
query7 0.02 0.01 0.01
query8 0.03 0.03 0.03
query9 0.58 0.51 0.54
query10 0.58 0.58 0.56
query11 0.14 0.09 0.10
query12 0.13 0.11 0.10
query13 0.58 0.58 0.57
query14 0.76 0.74 0.77
query15 0.83 0.81 0.82
query16 0.37 0.36 0.39
query17 0.98 1.03 1.01
query18 0.25 0.25 0.25
query19 1.86 1.75 1.69
query20 0.01 0.01 0.01
query21 15.41 0.61 0.59
query22 3.48 3.56 1.64
query23 17.28 1.00 0.86
query24 2.19 0.40 0.30
query25 0.22 0.11 0.07
query26 0.18 0.13 0.14
query27 0.03 0.03 0.03
query28 12.36 0.87 0.83
query29 12.54 3.20 3.28
query30 0.60 0.60 0.55
query31 2.80 0.34 0.34
query32 3.39 0.44 0.44
query33 2.78 2.81 2.82
query34 15.48 4.30 4.29
query35 4.33 4.30 4.31
query36 1.08 1.01 1.02
query37 0.07 0.06 0.05
query38 0.04 0.04 0.03
query39 0.02 0.02 0.02
query40 0.19 0.13 0.14
query41 0.08 0.02 0.02
query42 0.03 0.03 0.02
query43 0.02 0.02 0.02
Total cold run time: 105.43 s
Total hot run time: 29.78 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve](InPredict) enhance in predict with array type [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31828:
URL: https://github.com/apache/doris/pull/31828#issuecomment-1987192629
<details>
<summary>TPC-H: <b>Total hot run time: 37658 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f773e040b642075a1d78708083ad3f2a8e0721cb, data reload: false
------ Round 1 ----------------------------------
q1 17633 4118 4013 4013
q2 2037 145 138 138
q3 10571 968 924 924
q4 4658 937 960 937
q5 7607 2841 2979 2841
q6 177 122 123 122
q7 1306 825 836 825
q8 9478 1946 2081 1946
q9 7255 6480 6419 6419
q10 8214 2552 2563 2552
q11 417 221 211 211
q12 750 319 308 308
q13 17970 2941 2916 2916
q14 278 261 259 259
q15 487 459 444 444
q16 485 391 404 391
q17 955 865 847 847
q18 6625 6020 5827 5827
q19 1554 1521 1515 1515
q20 571 278 274 274
q21 7573 3676 3661 3661
q22 789 288 330 288
Total cold run time: 107390 ms
Total hot run time: 37658 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4012 3970 3964 3964
q2 311 218 229 218
q3 2939 2922 2907 2907
q4 1824 1781 1825 1781
q5 5205 5264 5236 5236
q6 202 116 114 114
q7 2249 1827 1814 1814
q8 3226 3261 3275 3261
q9 8531 8522 8479 8479
q10 6152 3745 3719 3719
q11 525 442 441 441
q12 696 533 545 533
q13 14842 2802 2830 2802
q14 281 255 256 255
q15 472 442 442 442
q16 454 419 411 411
q17 1692 1672 1641 1641
q18 7777 7377 7339 7339
q19 1632 1596 1602 1596
q20 1986 1717 1769 1717
q21 4990 4752 4697 4697
q22 531 470 476 470
Total cold run time: 70529 ms
Total hot run time: 53837 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org