You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "LemonLiTree (via GitHub)" <gi...@apache.org> on 2023/04/24 08:33:08 UTC
[GitHub] [doris] LemonLiTree opened a new pull request, #18999: [feature](geometry) Support geometry type
LemonLiTree opened a new pull request, #18999:
URL: https://github.com/apache/doris/pull/18999
Support geometry type, Geometry data is stored in wkb(Well-known_binary) format
mysql> create table geo_test
-> (
-> id int,
-> geodata geometry
-> )
-> DUPLICATE KEY(`id`)
-> DISTRIBUTED BY HASH(id)
-> PROPERTIES
-> (
-> "replication_num" = "1"
-> );
Query OK, 0 rows affected (0.05 sec)
mysql> insert into geo_test values (1,"POINT(1 1)");
Query OK, 1 row affected (0.05 sec)
{'label':'insert_21d0dd0e85dc4f74_9fbe459c06aaedbd', 'status':'VISIBLE', 'txnId':'15024'}
mysql> select * from geo_test;
+------+--------------------------------------------+
| id | geodata |
+------+--------------------------------------------+
| 1 | 0101000000000000000000f03f000000000000f03f |
+------+--------------------------------------------+
1 row in set (0.04 sec)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1622907428
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1524613930
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1622873825
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1521568825
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1520398893
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #18999:
URL: https://github.com/apache/doris/pull/18999#discussion_r1174965892
##########
be/src/runtime/geometry_value.h:
##########
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#ifndef DORIS_GEOMETRY_VALUE_H
+#define DORIS_GEOMETRY_VALUE_H
Review Comment:
warning: macro is not used [clang-diagnostic-unused-macros]
```cpp
#define DORIS_GEOMETRY_VALUE_H
^
```
##########
be/src/runtime/geometry_value.h:
##########
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#ifndef DORIS_GEOMETRY_VALUE_H
+#define DORIS_GEOMETRY_VALUE_H
+
+
+#include "geo/geo_tobinary.h"
+#include "common/status.h"
+
+
+namespace doris {
+struct GeometryBinaryValue {
+ static const int MAX_LENGTH = (1 << 30);
+
+ // default nullprt and size 0 for invalid or NULL value
+ const char* ptr = nullptr;
+ size_t len = 0;
+ std::stringstream result_stream;
+
+ GeometryBinaryValue() : ptr(nullptr), len(0) {}
+ GeometryBinaryValue(const std::string& s) { from_geometry_string(s.c_str(), s.length()); }
+ GeometryBinaryValue(const char* ptr, int len) { from_geometry_string(ptr, len); }
+
+ ~GeometryBinaryValue() {
+ delete[] ptr;
+ }
+
+ const char* value() { return ptr; }
Review Comment:
warning: method 'value' can be made const [readability-make-member-function-const]
```suggestion
const char* value() const { return ptr; }
```
##########
be/src/vec/data_types/data_type_geometry.h:
##########
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <ostream>
+
+#include "runtime/geometry_value.h"
+#include "vec/columns/column_string.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_string.h"
+
+namespace doris::vectorized {
+class DataTypeGeometry final : public IDataType {
+public:
+ using ColumnType = ColumnString;
+ static constexpr bool is_parametric = false;
+ using FieldType = GeometryField;
+
+ const char* get_family_name() const override { return "GEOMETRY"; }
+ TypeIndex get_type_id() const override { return TypeIndex::GEOMETRY; }
+ PrimitiveType get_type_as_primitive_type() const override { return TYPE_GEOMETRY; }
+ TPrimitiveType::type get_type_as_tprimitive_type() const override {
+ return TPrimitiveType::GEOMETRY;
+ }
+
+ int64_t get_uncompressed_serialized_bytes(const IColumn& column,
+ int data_version) const override;
+ char* serialize(const IColumn& column, char* buf, int data_version) const override;
+ const char* deserialize(const char* buf, IColumn* column, int data_version) const override;
+
+ MutableColumnPtr create_column() const override;
+
+ virtual Field get_default() const override {
Review Comment:
warning: 'virtual' is redundant since the function is already declared 'override' [modernize-use-override]
```suggestion
Field get_default() const override {
```
##########
be/src/runtime/geometry_value.h:
##########
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#ifndef DORIS_GEOMETRY_VALUE_H
+#define DORIS_GEOMETRY_VALUE_H
+
+
+#include "geo/geo_tobinary.h"
+#include "common/status.h"
+
+
+namespace doris {
+struct GeometryBinaryValue {
+ static const int MAX_LENGTH = (1 << 30);
+
+ // default nullprt and size 0 for invalid or NULL value
+ const char* ptr = nullptr;
+ size_t len = 0;
+ std::stringstream result_stream;
+
+ GeometryBinaryValue() : ptr(nullptr), len(0) {}
+ GeometryBinaryValue(const std::string& s) { from_geometry_string(s.c_str(), s.length()); }
+ GeometryBinaryValue(const char* ptr, int len) { from_geometry_string(ptr, len); }
+
+ ~GeometryBinaryValue() {
+ delete[] ptr;
+ }
+
+ const char* value() { return ptr; }
+
+ size_t size() { return len; }
Review Comment:
warning: method 'size' can be made const [readability-make-member-function-const]
```suggestion
size_t size() const { return len; }
```
##########
be/src/runtime/geometry_value.h:
##########
@@ -0,0 +1,60 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#ifndef DORIS_GEOMETRY_VALUE_H
+#define DORIS_GEOMETRY_VALUE_H
+
+
+#include "geo/geo_tobinary.h"
+#include "common/status.h"
+
+
+namespace doris {
+struct GeometryBinaryValue {
+ static const int MAX_LENGTH = (1 << 30);
+
+ // default nullprt and size 0 for invalid or NULL value
+ const char* ptr = nullptr;
+ size_t len = 0;
+ std::stringstream result_stream;
+
+ GeometryBinaryValue() : ptr(nullptr), len(0) {}
+ GeometryBinaryValue(const std::string& s) { from_geometry_string(s.c_str(), s.length()); }
+ GeometryBinaryValue(const char* ptr, int len) { from_geometry_string(ptr, len); }
+
+ ~GeometryBinaryValue() {
+ delete[] ptr;
+ }
+
+ const char* value() { return ptr; }
+
+ size_t size() { return len; }
+
+ void replace(char* ptr, int len) {
Review Comment:
warning: pointer parameter 'ptr' can be pointer to const [readability-non-const-parameter]
```suggestion
void replace(const char* ptr, int len) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] LemonLiTree closed pull request #18999: [feature](geometry) Support geometry type
Posted by "LemonLiTree (via GitHub)" <gi...@apache.org>.
LemonLiTree closed pull request #18999: [feature](geometry) Support geometry type
URL: https://github.com/apache/doris/pull/18999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1549295537
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1520020415
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1549427391
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1622961877
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 58.29 seconds
stream load tsv: 504 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 90.8 seconds inserted 10000000 Rows, about 110K ops/s
storage size: 17162198692 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230706123422_clickbench_pr_173280.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #18999: [feature](geometry) Support geometry type
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #18999:
URL: https://github.com/apache/doris/pull/18999#issuecomment-1622927100
(From new mechine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 53.67 seconds
stream load tsv: 514 seconds loaded 74807831229 Bytes, about 138 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 89.4 seconds inserted 10000000 Rows, about 111K ops/s
storage size: 17162260250 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230706114726_clickbench_pr_173256.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org