You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/06/24 01:54:30 UTC
[GitHub] [doris] xy720 opened a new pull request, #10385: [feature-wip](array-type) add function arrays_remove
xy720 opened a new pull request, #10385:
URL: https://github.com/apache/doris/pull/10385
# Proposed changes
Usage example:
array_remove remove all elements in array which is equal to the target.
```
mysql> set enable_vectorized_engine=true;
mysql> select k2, array_remove(k2, 1) from array_type_table_2;
+--------------------+-----------------------+
| k2 | array_remove(`k2`, 1) |
+--------------------+-----------------------+
| [1, 2, 3] | [2, 3] |
| [1, 2, 3] | [2, 3] |
| [1, 3] | [3] |
| [1, 3] | [3] |
| [1, 2, 1, 1] | [2] |
| [4, 1, 3, 1] | [4, 3] |
| [1, NULL, 3, NULL] | [NULL, 3, NULL] |
+--------------------+-----------------------+
7 rows in set (0.01 sec)
```
related to #xxx
## Checklist(Required)
1. Does it affect the original behavior: (Yes/No/I Don't know)
2. Has unit tests been added: (Yes/No/No Need)
3. Has document been added or modified: (Yes/No/No Need)
4. Does it need to update dependencies: (Yes/No)
5. Are there any changes that cannot be rolled back: (Yes/No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
xy720 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r911670411
##########
docs/en/docs/sql-manual/sql-functions/array-functions/array_remove.md:
##########
@@ -0,0 +1,79 @@
+---
+{
+ "title": "array_remove",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## array_remove
+
+### description
+
+#### Syntax
+
+```
+ARRAY<T> array_remove(ARRAY<T> arr, T)
+```
+
+Remove all elements that equal to element from array.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #10385:
URL: https://github.com/apache/doris/pull/10385#issuecomment-1180340826
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913605745
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
Review Comment:
```suggestion
MutableColumnPtr array_nested_column = nullptr;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #10385: [feature-wip](array-type) add function arrays_remove
Posted by GitBox <gi...@apache.org>.
adonis0147 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r905667287
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,363 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+
+
+// auto dst_null_column = ColumnUInt8::create(input_rows_count);
+// UInt8* dst_null_map = dst_null_column->get_data().data();
+// const UInt8* src_null_map = nullptr;
+// ColumnsWithTypeAndName args;
+// auto col_left = block.get_by_position(arguments[0]);
+// if (col_left.column->is_nullable()) {
+// auto null_col = check_and_get_column<ColumnNullable>(*col_left.column);
+// src_null_map = null_col->get_null_map_column().get_data().data();
+// args = {{null_col->get_nested_column_ptr(), remove_nullable(col_left.type),
+// col_left.name},
+// block.get_by_position(arguments[1])};
+// } else {
+// args = {col_left, block.get_by_position(arguments[1])};
+// }
+// auto res_column = _execute_non_nullable(args, input_rows_count, src_null_map);
Review Comment:
Useless code should be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 merged pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
xy720 merged PR #10385:
URL: https://github.com/apache/doris/pull/10385
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on PR #10385:
URL: https://github.com/apache/doris/pull/10385#issuecomment-1175861017
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #10385: [feature-wip](array-type) add function arrays_remove
Posted by GitBox <gi...@apache.org>.
xy720 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r909602988
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,363 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+
+
+// auto dst_null_column = ColumnUInt8::create(input_rows_count);
+// UInt8* dst_null_map = dst_null_column->get_data().data();
+// const UInt8* src_null_map = nullptr;
+// ColumnsWithTypeAndName args;
+// auto col_left = block.get_by_position(arguments[0]);
+// if (col_left.column->is_nullable()) {
+// auto null_col = check_and_get_column<ColumnNullable>(*col_left.column);
+// src_null_map = null_col->get_null_map_column().get_data().data();
+// args = {{null_col->get_nested_column_ptr(), remove_nullable(col_left.type),
+// col_left.name},
+// block.get_by_position(arguments[1])};
+// } else {
+// args = {col_left, block.get_by_position(arguments[1])};
+// }
+// auto res_column = _execute_non_nullable(args, input_rows_count, src_null_map);
Review Comment:
done
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,363 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+
+
+// auto dst_null_column = ColumnUInt8::create(input_rows_count);
+// UInt8* dst_null_map = dst_null_column->get_data().data();
+// const UInt8* src_null_map = nullptr;
+// ColumnsWithTypeAndName args;
+// auto col_left = block.get_by_position(arguments[0]);
+// if (col_left.column->is_nullable()) {
+// auto null_col = check_and_get_column<ColumnNullable>(*col_left.column);
+// src_null_map = null_col->get_null_map_column().get_data().data();
+// args = {{null_col->get_nested_column_ptr(), remove_nullable(col_left.type),
+// col_left.name},
+// block.get_by_position(arguments[1])};
+// } else {
+// args = {col_left, block.get_by_position(arguments[1])};
+// }
+// auto res_column = _execute_non_nullable(args, input_rows_count, src_null_map);
+
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ LOG(INFO) << "left type is " << args[0].type->get_name();
+ LOG(INFO) << "right type is " << args[1].type->get_name();
+ LOG(INFO) << "right column type is " << args[1].column->get_name();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #10385: [feature-wip](array-type) add function arrays_remove
Posted by GitBox <gi...@apache.org>.
xy720 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r909610303
##########
be/src/vec/utils/util.hpp:
##########
@@ -38,6 +40,12 @@ class VectorizedUtils {
ColumnsWithTypeAndName columns_with_type_and_name;
for (const auto& tuple_desc : row_desc.tuple_descriptors()) {
for (const auto& slot_desc : tuple_desc->slots()) {
+ LOG(INFO) << "slot type is " << slot_desc->get_data_type_ptr()->get_name();
Review Comment:
Ha ha, I forget to remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
xy720 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913922416
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r911573511
##########
docs/en/docs/sql-manual/sql-functions/array-functions/array_remove.md:
##########
@@ -0,0 +1,79 @@
+---
+{
+ "title": "array_remove",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## array_remove
+
+### description
+
+#### Syntax
+
+```
+ARRAY<T> array_remove(ARRAY<T> arr, T)
Review Comment:
```suggestion
ARRAY<T> array_remove(ARRAY<T> arr, T val)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] carlvinhust2012 commented on a diff in pull request #10385: [feature-wip](array-type) add function arrays_remove
Posted by GitBox <gi...@apache.org>.
carlvinhust2012 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r909605611
##########
be/src/vec/utils/util.hpp:
##########
@@ -38,6 +40,12 @@ class VectorizedUtils {
ColumnsWithTypeAndName columns_with_type_and_name;
for (const auto& tuple_desc : row_desc.tuple_descriptors()) {
for (const auto& slot_desc : tuple_desc->slots()) {
+ LOG(INFO) << "slot type is " << slot_desc->get_data_type_ptr()->get_name();
Review Comment:
is this log necessary？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r911663562
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,370 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const UInt8* arr_null_map, const IColumn& target,
+ const UInt8* nested_null_map, UInt8* dst_null_map, UInt8* target_null_map) {
+ // check array nested column type and get data
+ auto& src_column = reinterpret_cast<const NestedColumnType&>(nested_column);
+ const auto& src_data = src_column.get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(target).get_data();
+
+ // prepare dst array
+ auto dst = ColumnArray::create(
+ ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create()));
+ auto& dst_offsets = dst->get_offsets();
+ dst_offsets.reserve(offsets.size());
+
+ // prepare dst nested nullable column
+ auto& dst_column = reinterpret_cast<ColumnNullable&>(dst->get_data());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ // case: array:NULL, target:1 ==> NULL
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (!nested_null_map && target_null_map[row]) {
+ // case: array:[1,2], target:Null ==> [1,2]
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ continue;
+ }
+
+ if (nested_null_map) {
+ if (nested_null_map[off + pos] && !target_null_map[row]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_column.insert_data(nullptr, 0);
+ continue;
+ } else if (!nested_null_map[off + pos] && target_null_map[row]) {
+ // case: array:[1,2], target:Null ==> [1,2]
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ continue;
+ } else if (nested_null_map[off + pos] && target_null_map[row]) {
+ // case: array:[Null], target:Null ==> []
+ ++count;
+ continue;
+ }
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const UInt8* arr_null_map, const IColumn& target,
+ const UInt8* nested_null_map, UInt8* dst_null_map, UInt8* target_null_map) {
+ // check array nested column type and get data
+ auto& src_column = reinterpret_cast<const ColumnString&>(nested_column);
+
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(target).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(target).get_chars();
+
+ // prepare dst array
+ auto dst = ColumnArray::create(
Review Comment:
need to consider nested column nullable or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #10385:
URL: https://github.com/apache/doris/pull/10385#issuecomment-1185383258
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913606190
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
Review Comment:
```suggestion
array_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913614152
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
Review Comment:
```suggestion
if (nested_null_map) {
dst_null_map->push_back(0);
}
dst_data.push_back(src_data[off + pos]);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
xy720 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913053590
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,370 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const UInt8* arr_null_map, const IColumn& target,
+ const UInt8* nested_null_map, UInt8* dst_null_map, UInt8* target_null_map) {
+ // check array nested column type and get data
+ auto& src_column = reinterpret_cast<const NestedColumnType&>(nested_column);
+ const auto& src_data = src_column.get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(target).get_data();
+
+ // prepare dst array
+ auto dst = ColumnArray::create(
+ ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create()));
+ auto& dst_offsets = dst->get_offsets();
+ dst_offsets.reserve(offsets.size());
+
+ // prepare dst nested nullable column
+ auto& dst_column = reinterpret_cast<ColumnNullable&>(dst->get_data());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ // case: array:NULL, target:1 ==> NULL
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (!nested_null_map && target_null_map[row]) {
+ // case: array:[1,2], target:Null ==> [1,2]
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ continue;
+ }
+
+ if (nested_null_map) {
+ if (nested_null_map[off + pos] && !target_null_map[row]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_column.insert_data(nullptr, 0);
+ continue;
+ } else if (!nested_null_map[off + pos] && target_null_map[row]) {
+ // case: array:[1,2], target:Null ==> [1,2]
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ continue;
+ } else if (nested_null_map[off + pos] && target_null_map[row]) {
+ // case: array:[Null], target:Null ==> []
+ ++count;
+ continue;
+ }
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ dst_column.insert_from_not_nullable(src_column, off + pos);
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const UInt8* arr_null_map, const IColumn& target,
+ const UInt8* nested_null_map, UInt8* dst_null_map, UInt8* target_null_map) {
+ // check array nested column type and get data
+ auto& src_column = reinterpret_cast<const ColumnString&>(nested_column);
+
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(target).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(target).get_chars();
+
+ // prepare dst array
+ auto dst = ColumnArray::create(
Review Comment:
Ok. I have refactored the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913622165
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
+ dst_data.push_back(src_data[off + pos]);
+ dst_null_map->push_back(0);
+ } else {
+ dst_data.push_back(src_data[off + pos]);
+ }
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ auto dst = ColumnArray::create(std::move(array_nested_column), std::move(dst_offsets_column));
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(right_column).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(right_column).get_chars();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_offs = reinterpret_cast<ColumnString&>(*dst_column).get_offsets();
+ auto& dst_chars = reinterpret_cast<ColumnString&>(*dst_column).get_chars();
+ dst_offs.reserve(src_offs.back());
+ dst_chars.reserve(src_offs.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:'str' ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t target_off = target_offs[row - 1];
+ size_t target_len = target_offs[row] - target_off;
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:'str' ==> [Null]
+ dst_chars.push_back(0);
+ dst_offs.push_back(dst_offs.back() + 1);
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ size_t src_pos = src_offs[pos + off - 1];
+ size_t src_len = src_offs[pos + off] - src_pos;
+ const char* src_raw_v = reinterpret_cast<const char*>(&src_chars[src_pos]);
+ const char* target_raw_v = reinterpret_cast<const char*>(&target_chars[target_off]);
+
+ if (std::string_view(src_raw_v, src_len) == std::string_view(target_raw_v, target_len)) {
+ ++count;
+ } else {
+ if (src_len == 1) {
Review Comment:
no need to process empty string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913615831
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
+ dst_data.push_back(src_data[off + pos]);
+ dst_null_map->push_back(0);
+ } else {
+ dst_data.push_back(src_data[off + pos]);
+ }
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ auto dst = ColumnArray::create(std::move(array_nested_column), std::move(dst_offsets_column));
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(right_column).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(right_column).get_chars();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_offs = reinterpret_cast<ColumnString&>(*dst_column).get_offsets();
+ auto& dst_chars = reinterpret_cast<ColumnString&>(*dst_column).get_chars();
+ dst_offs.reserve(src_offs.back());
Review Comment:
```suggestion
dst_offs.reserve(src_offs.size());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #10385: [feature-wip](array-type) add function arrays_remove
Posted by GitBox <gi...@apache.org>.
adonis0147 commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r905668739
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,363 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+
+
+// auto dst_null_column = ColumnUInt8::create(input_rows_count);
+// UInt8* dst_null_map = dst_null_column->get_data().data();
+// const UInt8* src_null_map = nullptr;
+// ColumnsWithTypeAndName args;
+// auto col_left = block.get_by_position(arguments[0]);
+// if (col_left.column->is_nullable()) {
+// auto null_col = check_and_get_column<ColumnNullable>(*col_left.column);
+// src_null_map = null_col->get_null_map_column().get_data().data();
+// args = {{null_col->get_nested_column_ptr(), remove_nullable(col_left.type),
+// col_left.name},
+// block.get_by_position(arguments[1])};
+// } else {
+// args = {col_left, block.get_by_position(arguments[1])};
+// }
+// auto res_column = _execute_non_nullable(args, input_rows_count, src_null_map);
+
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ LOG(INFO) << "left type is " << args[0].type->get_name();
+ LOG(INFO) << "right type is " << args[1].type->get_name();
+ LOG(INFO) << "right column type is " << args[1].column->get_name();
Review Comment:
Logging in IO path should be simplified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #10385:
URL: https://github.com/apache/doris/pull/10385#issuecomment-1180340795
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r911573816
##########
docs/en/docs/sql-manual/sql-functions/array-functions/array_remove.md:
##########
@@ -0,0 +1,79 @@
+---
+{
+ "title": "array_remove",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## array_remove
+
+### description
+
+#### Syntax
+
+```
+ARRAY<T> array_remove(ARRAY<T> arr, T)
+```
+
+Remove all elements that equal to element from array.
Review Comment:
Add more cases, like: If arr is NULL or val is NULL, how array_remove works?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yangzhg commented on pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
yangzhg commented on PR #10385:
URL: https://github.com/apache/doris/pull/10385#issuecomment-1178425949
check the p0 reg and BE Code Formatter /
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913599406
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
+ dst_data.push_back(src_data[off + pos]);
+ dst_null_map->push_back(0);
+ } else {
+ dst_data.push_back(src_data[off + pos]);
+ }
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ auto dst = ColumnArray::create(std::move(array_nested_column), std::move(dst_offsets_column));
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(right_column).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(right_column).get_chars();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_offs = reinterpret_cast<ColumnString&>(*dst_column).get_offsets();
+ auto& dst_chars = reinterpret_cast<ColumnString&>(*dst_column).get_chars();
+ dst_offs.reserve(src_offs.back());
+ dst_chars.reserve(src_offs.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:'str' ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t target_off = target_offs[row - 1];
+ size_t target_len = target_offs[row] - target_off;
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:'str' ==> [Null]
+ dst_chars.push_back(0);
+ dst_offs.push_back(dst_offs.back() + 1);
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ size_t src_pos = src_offs[pos + off - 1];
+ size_t src_len = src_offs[pos + off] - src_pos;
+ const char* src_raw_v = reinterpret_cast<const char*>(&src_chars[src_pos]);
+ const char* target_raw_v = reinterpret_cast<const char*>(&target_chars[target_off]);
+
+ if (std::string_view(src_raw_v, src_len) == std::string_view(target_raw_v, target_len)) {
+ ++count;
+ } else {
+ if (src_len == 1) {
+ // case: array:[''], target:'str' ==> ['']
+ dst_chars.push_back(0);
+ dst_offs.push_back(dst_offs.back() + 1);
+ } else {
+ const size_t old_size = dst_chars.size();
+ const size_t new_size = old_size + src_len;
+ dst_chars.resize(new_size);
+ memcpy(&dst_chars[old_size], &src_chars[src_pos], src_len);
+ dst_offs.push_back(new_size);
+ }
+
+ if (nested_null_map) {
+ dst_null_map->push_back(0);
+ }
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ auto dst = ColumnArray::create(std::move(array_nested_column), std::move(dst_offsets_column));
+ return dst;
+ }
+
+ template <typename NestedColumnType>
+ ColumnPtr _execute_number_expanded(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ if (check_column<ColumnUInt8>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnUInt8>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnInt8>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnInt8>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnInt16>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnInt16>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnInt32>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnInt32>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnInt64>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnInt64>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnInt128>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnInt128>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnFloat32>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnFloat32>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (check_column<ColumnFloat64>(right_column)) {
+ return _execute_number<NestedColumnType, ColumnFloat64>(offsets, nested_column,
+ right_column, nested_null_map);
+ } else if (right_column.is_date_type()) {
Review Comment:
put this line before check_column<ColumnInt64>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #10385: [feature-wip](array-type) add function array_remove
Posted by GitBox <gi...@apache.org>.
cambyzju commented on code in PR #10385:
URL: https://github.com/apache/doris/pull/10385#discussion_r913619629
##########
be/src/vec/functions/array/function_array_remove.h:
##########
@@ -0,0 +1,345 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayRemove : public IFunction {
+public:
+ static constexpr auto name = "array_remove";
+ static FunctionPtr create() { return std::make_shared<FunctionArrayRemove>(); }
+
+ /// Get function name.
+ String get_name() const override { return name; }
+
+ bool is_variadic() const override { return false; }
+
+ size_t get_number_of_arguments() const override { return 2; }
+
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ DCHECK(is_array(arguments[0]))
+ << "First argument for function: " << name << " should be DataTypeArray but it has type "
+ << arguments[0]->get_name() << ".";
+ return arguments[0];
+ }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) override {
+ // For default implementation of nulls args
+ ColumnsWithTypeAndName args = {
+ block.get_by_position(arguments[0]), block.get_by_position(arguments[1])
+ };
+
+ auto res_column = _execute_non_nullable(args, input_rows_count);
+ if (!res_column) {
+ return Status::RuntimeError(
+ fmt::format("unsupported types for function {}({}, {})", get_name(),
+ block.get_by_position(arguments[0]).type->get_name(),
+ block.get_by_position(arguments[1]).type->get_name()));
+ }
+ DCHECK_EQ(args[0].column->size(), res_column->size());
+ block.replace_by_position(result, std::move(res_column));
+ return Status::OK();
+ }
+
+private:
+
+ template <typename NestedColumnType, typename RightColumnType>
+ ColumnPtr _execute_number(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_data = reinterpret_cast<const NestedColumnType&>(nested_column).get_data();
+
+ // check target column type and get data
+ const auto& target_data = reinterpret_cast<const RightColumnType&>(right_column).get_data();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_data = reinterpret_cast<NestedColumnType&>(*dst_column).get_data();
+ dst_data.reserve(offsets.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:1 ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:1 ==> [Null]
+ dst_data.push_back(typename NestedColumnType::value_type());
+ dst_null_map->push_back(1);
+ continue;
+ }
+
+ if (src_data[off + pos] == target_data[row]) {
+ ++count;
+ } else {
+ if (nested_null_map) {
+ dst_data.push_back(src_data[off + pos]);
+ dst_null_map->push_back(0);
+ } else {
+ dst_data.push_back(src_data[off + pos]);
+ }
+ }
+ }
+
+ cur += len - count;
+ dst_offsets.push_back(cur);
+ }
+
+ auto dst = ColumnArray::create(std::move(array_nested_column), std::move(dst_offsets_column));
+ return dst;
+ }
+
+ ColumnPtr _execute_string(const ColumnArray::Offsets& offsets, const IColumn& nested_column,
+ const IColumn& right_column, const UInt8* nested_null_map) {
+ // check array nested column type and get data
+ const auto& src_offs = reinterpret_cast<const ColumnString&>(nested_column).get_offsets();
+ const auto& src_chars = reinterpret_cast<const ColumnString&>(nested_column).get_chars();
+
+ // check right column type and get data
+ const auto& target_offs = reinterpret_cast<const ColumnString&>(right_column).get_offsets();
+ const auto& target_chars = reinterpret_cast<const ColumnString&>(right_column).get_chars();
+
+ PaddedPODArray<UInt8>* dst_null_map = nullptr;
+ ColumnPtr array_nested_column = nullptr;
+ IColumn* dst_column;
+ if (nested_null_map) {
+ auto dst_nested_column = ColumnNullable::create(nested_column.clone_empty(), ColumnUInt8::create());
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column->get_nested_column_ptr();
+ dst_null_map = &dst_nested_column->get_null_map_data();
+ dst_null_map->reserve(offsets.back());
+ } else {
+ auto dst_nested_column = nested_column.clone_empty();
+ array_nested_column = dst_nested_column->get_ptr();
+ dst_column = dst_nested_column;
+ }
+
+ auto& dst_offs = reinterpret_cast<ColumnString&>(*dst_column).get_offsets();
+ auto& dst_chars = reinterpret_cast<ColumnString&>(*dst_column).get_chars();
+ dst_offs.reserve(src_offs.back());
+ dst_chars.reserve(src_offs.back());
+
+ auto dst_offsets_column = ColumnArray::ColumnOffsets::create();
+ auto& dst_offsets = dst_offsets_column->get_data();
+ dst_offsets.reserve(offsets.size());
+
+ size_t cur = 0;
+ for (size_t row = 0; row < offsets.size(); ++row) {
+ size_t off = offsets[row - 1];
+ size_t len = offsets[row] - off;
+
+ if (len == 0) {
+ // case: array:[], target:'str' ==> []
+ dst_offsets.push_back(cur);
+ continue;
+ }
+
+ size_t target_off = target_offs[row - 1];
+ size_t target_len = target_offs[row] - target_off;
+
+ size_t count = 0;
+ for (size_t pos = 0; pos < len; ++pos) {
+ if (nested_null_map && nested_null_map[off + pos]) {
+ // case: array:[Null], target:'str' ==> [Null]
+ dst_chars.push_back(0);
Review Comment:
no need to push_back(0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org