You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "gitccl (via GitHub)" <gi...@apache.org> on 2023/03/05 16:11:12 UTC
[GitHub] [doris] gitccl opened a new pull request, #17436: [Feature](array-function) Support array_concat function
gitccl opened a new pull request, #17436:
URL: https://github.com/apache/doris/pull/17436
# Proposed changes
Issue Number: close #17307
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [X] Has document been added or modified
* [ ] Does it need to update dependencies
* [X] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1457414455
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on code in PR #17436:
URL: https://github.com/apache/doris/pull/17436#discussion_r1125696675
##########
be/src/vec/functions/array/function_array_concat.cpp:
##########
@@ -0,0 +1,84 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include <cstddef>
+
+#include "vec/columns/column_array.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/functions/function.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+// array_concat([1, 2], [7, 8], [5, 6]) -> [1, 2, 7, 8, 5, 6]
+class FunctionArrayConcat : public IFunction {
+public:
+ static constexpr auto name = "array_concat";
+
+ static FunctionPtr create() { return std::make_shared<FunctionArrayConcat>(); }
+
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return true; }
+
+ size_t get_number_of_arguments() const override { return 1; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(arguments.size() > 0)
+ << "function: " << get_name() << ", arguments should not be empty";
+ for (const auto& arg : arguments) {
+ DCHECK(is_array(arg)) << "argument for function array_concat should be DataTypeArray"
+ << " and argument is " << arg->get_name();
+ }
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ const size_t result, size_t input_rows_count) override {
+ DataTypePtr column_type = block.get_by_position(arguments[0]).type;
+ auto nested_type = assert_cast<const DataTypeArray&>(*column_type).get_nested_type();
+ auto result_column = ColumnArray::create(nested_type->create_column(),
+ ColumnArray::ColumnOffsets::create());
+ IColumn& result_nested_col = result_column->get_data();
Review Comment:
we could do `reserve` for result_nested_col for more efficiency
##########
be/src/vec/functions/array/function_array_concat.cpp:
##########
@@ -0,0 +1,84 @@
+// Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
also need add this doc to sidebar `docs/sidebars.json`
##########
regression-test/suites/query_p0/sql_functions/array_functions/test_array_functions_by_literal.groovy:
##########
@@ -223,6 +223,14 @@ suite("test_array_functions_by_literal") {
qt_sql """select array_apply(array(cast (24.99 as decimal(10,3)),cast (25.99 as decimal(10,3))), ">", '25')"""
qt_sql """select array_apply(array(cast (24.99 as decimal(10,3)),cast (25.99 as decimal(10,3))), "!=", '25.99')"""
+ qt_sql "select array_concat([1, 2, 3], [2, 3, 4], [8, 1, 2], [9])"
+ qt_sql "select array_concat([12, 23], [25, null], [null], [66])"
+ qt_sql "select array_concat([1.2, 1.8], [9.0, 2.2], [2.8])"
+ qt_sql "select array_concat(['aaa', null], ['bbb', 'fff'], [null, 'ccc'])"
Review Comment:
add a null case like array_concat(null, [1, 2, 3], null)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1458104164
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gitccl commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "gitccl (via GitHub)" <gi...@apache.org>.
gitccl commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1458098441
> This pr has conflicts, please rebase
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455135119
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gitccl commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "gitccl (via GitHub)" <gi...@apache.org>.
gitccl commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455393697
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gitccl commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "gitccl (via GitHub)" <gi...@apache.org>.
gitccl commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455134897
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1457905530
This pr has conflicts, please rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455446817
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1457414100
run beut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455395561
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #17436: [Feature](array-function) Support array_concat function
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #17436:
URL: https://github.com/apache/doris/pull/17436
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on code in PR #17436:
URL: https://github.com/apache/doris/pull/17436#discussion_r1125868175
##########
be/src/vec/functions/array/function_array_concat.cpp:
##########
@@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include <cstddef>
+
+#include "vec/columns/column_array.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/functions/function.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+// array_concat([1, 2], [7, 8], [5, 6]) -> [1, 2, 7, 8, 5, 6]
+class FunctionArrayConcat : public IFunction {
+public:
+ static constexpr auto name = "array_concat";
+
+ static FunctionPtr create() { return std::make_shared<FunctionArrayConcat>(); }
+
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return true; }
+
+ size_t get_number_of_arguments() const override { return 1; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(arguments.size() > 0)
+ << "function: " << get_name() << ", arguments should not be empty";
+ for (const auto& arg : arguments) {
+ DCHECK(is_array(arg)) << "argument for function array_concat should be DataTypeArray"
+ << " and argument is " << arg->get_name();
+ }
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ const size_t result, size_t input_rows_count) override {
+ DataTypePtr column_type = block.get_by_position(arguments[0]).type;
+ auto nested_type = assert_cast<const DataTypeArray&>(*column_type).get_nested_type();
+ auto result_column = ColumnArray::create(nested_type->create_column(),
+ ColumnArray::ColumnOffsets::create());
+ IColumn& result_nested_col = result_column->get_data();
+ ColumnArray::Offsets64& column_offsets = result_column->get_offsets();
+ column_offsets.resize(input_rows_count);
+
+ size_t total_size = 0;
+ for (size_t col : arguments) {
+ ColumnPtr src_column =
+ block.get_by_position(col).column->convert_to_full_column_if_const();
+ const auto& src_column_array = check_and_get_column<ColumnArray>(*src_column);
Review Comment:
we should get the nested column of array, and caculate it's size, other wise, it's the row size of array not the inner nested row size of the array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1459165726
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gitccl commented on a diff in pull request #17436: [Feature](array-function) Support array_concat function
Posted by "gitccl (via GitHub)" <gi...@apache.org>.
gitccl commented on code in PR #17436:
URL: https://github.com/apache/doris/pull/17436#discussion_r1125904975
##########
be/src/vec/functions/array/function_array_concat.cpp:
##########
@@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include <cstddef>
+
+#include "vec/columns/column_array.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/functions/function.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+// array_concat([1, 2], [7, 8], [5, 6]) -> [1, 2, 7, 8, 5, 6]
+class FunctionArrayConcat : public IFunction {
+public:
+ static constexpr auto name = "array_concat";
+
+ static FunctionPtr create() { return std::make_shared<FunctionArrayConcat>(); }
+
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return true; }
+
+ size_t get_number_of_arguments() const override { return 1; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(arguments.size() > 0)
+ << "function: " << get_name() << ", arguments should not be empty";
+ for (const auto& arg : arguments) {
+ DCHECK(is_array(arg)) << "argument for function array_concat should be DataTypeArray"
+ << " and argument is " << arg->get_name();
+ }
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ const size_t result, size_t input_rows_count) override {
+ DataTypePtr column_type = block.get_by_position(arguments[0]).type;
+ auto nested_type = assert_cast<const DataTypeArray&>(*column_type).get_nested_type();
+ auto result_column = ColumnArray::create(nested_type->create_column(),
+ ColumnArray::ColumnOffsets::create());
+ IColumn& result_nested_col = result_column->get_data();
+ ColumnArray::Offsets64& column_offsets = result_column->get_offsets();
+ column_offsets.resize(input_rows_count);
+
+ size_t total_size = 0;
+ for (size_t col : arguments) {
+ ColumnPtr src_column =
+ block.get_by_position(col).column->convert_to_full_column_if_const();
+ const auto& src_column_array = check_and_get_column<ColumnArray>(*src_column);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17436: [Feature](array-function) Support array_concat function
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17436:
URL: https://github.com/apache/doris/pull/17436#issuecomment-1455446685
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org