You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mrhhsg (via GitHub)" <gi...@apache.org> on 2023/11/01 02:38:22 UTC
[PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
mrhhsg opened a new pull request, #26191:
URL: https://github.com/apache/doris/pull/26191
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1791676407
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1811912204
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812097035
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378865207
##########
be/src/olap/in_list_predicate.h:
##########
@@ -395,6 +371,16 @@ class InListPredicateBase : public ColumnPredicate {
if (bf->test_bytes(reinterpret_cast<const char*>(value), sizeof(uint24_t))) {
return true;
}
+ } else if constexpr (Type == PrimitiveType::TYPE_DECIMALV2) {
+ // DecimalV2 using decimal12_t in bloom filter in storage layer,
+ // should convert value to decimal12_t
+ const T* value = (const T*)(iter->get_value());
+ decimal12_t decimal12_t_val(value->int_value(), value->frac_value());
+ if (bf->test_bytes(
+ const_cast<char*>(reinterpret_cast<const char*>(&decimal12_t_val)),
+ sizeof(decimal12_t))) {
+ return true;
Review Comment:
what about datev1 and datetimev1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809340662
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1811910306
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378874597
##########
regression-test/suites/datatype_p0/scalar_types/load.groovy:
##########
@@ -90,8 +90,8 @@ suite("test_scalar_types_load", "p0") {
`c_float` float NULL,
`c_double` double NULL,
`c_decimal` decimal(20, 3) NULL,
- `c_date` date NULL,
- `c_datetime` datetime NULL,
+ `c_date` datev1 NULL,
+ `c_datetime` datetimev1 NULL,
`c_datev2` datev2 NULL,
Review Comment:
we should add more test cases including:
1. in list predicate
2. delete predicate
3. comparison predicate
4. is null predicate
test index:
bloom filter index
min max index
bitmap index
test key range
test all data types
test runtime filter // maybe very difficult
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1788529915
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1788522388
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809312759
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1788318333
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378861254
##########
be/src/olap/comparison_predicate.h:
##########
@@ -232,8 +251,16 @@ class ComparisonPredicateBase : public ColumnPredicate {
if constexpr (std::is_same_v<T, StringRef>) {
return bf->test_bytes(_value.data, _value.size);
} else {
- return bf->test_bytes(const_cast<char*>(reinterpret_cast<const char*>(&_value)),
- sizeof(WarpperFieldType));
+ // DecimalV2 using decimal12_t in bloom filter, should convert value to decimal12_t
+ if constexpr (Type == PrimitiveType::TYPE_DECIMALV2) {
+ decimal12_t decimal12_t_val(_value.int_value(), _value.frac_value());
+ return bf->test_bytes(
Review Comment:
what about datev1 and datetimev1, do they also need convert to storage type here? if not please add comment here to explain the reason.
##########
be/src/olap/comparison_predicate.h:
##########
@@ -232,8 +251,16 @@ class ComparisonPredicateBase : public ColumnPredicate {
if constexpr (std::is_same_v<T, StringRef>) {
return bf->test_bytes(_value.data, _value.size);
} else {
- return bf->test_bytes(const_cast<char*>(reinterpret_cast<const char*>(&_value)),
- sizeof(WarpperFieldType));
+ // DecimalV2 using decimal12_t in bloom filter, should convert value to decimal12_t
+ if constexpr (Type == PrimitiveType::TYPE_DECIMALV2) {
+ decimal12_t decimal12_t_val(_value.int_value(), _value.frac_value());
+ return bf->test_bytes(
Review Comment:
what about datev1 and datetimev1, do they also need convert to storage type here? if not please add comment here to explain the reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790594944
TeamCity be ut coverage result:
Function Coverage: 37.37% (8481/22697)
Line Coverage: 29.73% (68589/230698)
Region Coverage: 28.39% (35496/125045)
Branch Coverage: 25.12% (18116/72116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6cedf0827aba8717ff96a20fab7d4c77d3dc0198_6cedf0827aba8717ff96a20fab7d4c77d3dc0198/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1386208584
##########
be/src/vec/columns/column_vector.h:
##########
@@ -198,10 +198,10 @@ class ColumnVector final : public COWHelper<ColumnVectorHelper, ColumnVector<T>>
void insert_date_column(const char* data_ptr, size_t num) {
data.reserve(data.size() + num);
- size_t input_value_size = sizeof(uint24_t);
+ constexpr size_t input_value_size = sizeof(uint24_t);
for (int i = 0; i < num; i++) {
- uint64_t val = 0;
+ uint24_t val = 0;
Review Comment:
我估计啊,你这么改性能不一定高, 这个24bit 不对齐,可能都不如直接uint32_t
##########
be/src/vec/columns/column_vector.h:
##########
@@ -198,10 +198,10 @@ class ColumnVector final : public COWHelper<ColumnVectorHelper, ColumnVector<T>>
void insert_date_column(const char* data_ptr, size_t num) {
data.reserve(data.size() + num);
- size_t input_value_size = sizeof(uint24_t);
+ constexpr size_t input_value_size = sizeof(uint24_t);
for (int i = 0; i < num; i++) {
- uint64_t val = 0;
+ uint24_t val = 0;
Review Comment:
我估计啊,你这么改性能不一定高, 这个24bit 不对齐,可能都不如直接uint32_t
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809318693
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809421548
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit b8f25b740c1c70e22e69804be79b20b0142f6ae8, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5282 5199 5170 5170
q2 364 153 159 153
q3 2039 2029 1983 1983
q4 1412 1373 1366 1366
q5 3958 3966 3999 3966
q6 258 133 126 126
q7 1494 875 885 875
q8 2778 2797 2760 2760
q9 9569 9415 9453 9415
q10 3457 3560 3554 3554
q11 397 261 250 250
q12 444 301 309 301
q13 4571 4123 4121 4121
q14 333 292 290 290
q15 623 558 539 539
q16 678 594 582 582
q17 1122 1072 1073 1072
q18 8110 7651 7734 7651
q19 1677 1691 1685 1685
q20 532 284 296 284
q21 4706 4387 4394 4387
q22 513 411 414 411
Total cold run time: 54317 ms
Total hot run time: 50941 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5048 5123 5116 5116
q2 331 227 241 227
q3 4106 3953 4030 3953
q4 2848 2760 2782 2760
q5 9693 9685 9746 9685
q6 243 123 123 123
q7 3076 2473 2540 2473
q8 4852 4863 4901 4863
q9 13041 13001 12930 12930
q10 4057 4187 4149 4149
q11 733 667 672 667
q12 998 893 852 852
q13 4328 3885 3908 3885
q14 368 345 341 341
q15 595 547 550 547
q16 744 684 727 684
q17 3911 3885 3962 3885
q18 9629 9489 9463 9463
q19 1844 1778 1757 1757
q20 2385 2054 2030 2030
q21 8992 8903 8869 8869
q22 946 846 874 846
Total cold run time: 82768 ms
Total hot run time: 80105 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1807831374
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1392564155
##########
be/src/vec/columns/predicate_column.h:
##########
@@ -447,13 +459,9 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<
// DateV1 and DateTimeV1 is special, its storage format is different from compute format
// should convert here.
if constexpr (Type == TYPE_DATE || Type == TYPE_DATETIME) {
- if constexpr (std::is_same_v<T, uint32_t>) {
- insert_date_to_res_column(sel, sel_size, column);
- } else if constexpr (std::is_same_v<T, uint64_t>) {
- insert_datetime_to_res_column(sel, sel_size, column);
- } else {
- LOG(FATAL) << "not reachable";
- }
+ insert_datetime_to_res_column(sel, sel_size, column);
Review Comment:
If datev1 using the same type in predicate and compute layer, why do we need use a specific method here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812183500
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790497656
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790648620
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.78 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161981252 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790500292
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1393539828
##########
be/src/exprs/bloom_filter_func.h:
##########
@@ -416,37 +416,10 @@ struct FixedStringFindOp : public StringFindOp {
}
};
-struct DateTimeFindOp : public CommonFindOp<VecDateTimeValue> {
- bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- VecDateTimeValue value;
- value.from_olap_datetime(*reinterpret_cast<const uint64_t*>(data));
- return bloom_filter.test(Slice((char*)&value, sizeof(VecDateTimeValue)));
- }
-};
-
-// avoid violating C/C++ aliasing rules.
-// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101684
-
-struct DateFindOp : public CommonFindOp<VecDateTimeValue> {
- bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- uint24_t date = *static_cast<const uint24_t*>(data);
- uint64_t value = uint32_t(date);
-
- VecDateTimeValue date_value;
- date_value.from_olap_date(value);
-
- return bloom_filter.test(Slice((char*)&date_value, sizeof(VecDateTimeValue)));
- }
-};
-
struct DecimalV2FindOp : public CommonFindOp<DecimalV2Value> {
bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- auto packed_decimal = *static_cast<const decimal12_t*>(data);
- DecimalV2Value value;
- int64_t int_value = packed_decimal.integer;
- int32_t frac_value = packed_decimal.fraction;
- value.from_olap_decimal(int_value, frac_value);
-
+ // Predicate column using decimalv2value as column value type, so not need convert here
+ DecimalV2Value value = *static_cast<const DecimalV2Value*>(data);
constexpr int decimal_value_sz = sizeof(DecimalV2Value);
char data_bytes[decimal_value_sz];
memcpy(&data_bytes, &value, decimal_value_sz);
Review Comment:
remove duplicate memcpy code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812238164
TeamCity be ut coverage result:
Function Coverage: 36.82% (8423/22879)
Line Coverage: 29.29% (68395/233517)
Region Coverage: 27.90% (35371/126787)
Branch Coverage: 24.66% (18077/73292)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b1b0c4a3861d8d4d743abb159ae14f844c585c5d_b1b0c4a3861d8d4d743abb159ae14f844c585c5d/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809429606
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.33 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162099424 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809346602
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790594728
TeamCity be ut coverage result:
Function Coverage: 37.36% (8481/22698)
Line Coverage: 29.73% (68583/230712)
Region Coverage: 28.38% (35490/125055)
Branch Coverage: 25.11% (18111/72116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cfc940efde290134bcc3a5b74b759683c64a4289_cfc940efde290134bcc3a5b74b759683c64a4289/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1791759364
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.92 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162274849 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378871450
##########
be/src/olap/predicate_creator.h:
##########
@@ -49,7 +49,7 @@ class PredicateCreator {
template <PrimitiveType Type, PredicateType PT, typename ConditionType>
class IntegerPredicateCreator : public PredicateCreator<ConditionType> {
public:
Review Comment:
maybe not right here. because many predicate parse the value from string, and olap scannode convert the predicate value to a string and predicate creator parse the value from string.
If you do not change olap scannode code, then I think you could not parse the value correctly.
I think, maybe we should delete predicate creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1800958519
TeamCity be ut coverage result:
Function Coverage: 37.02% (8395/22675)
Line Coverage: 29.44% (67948/230781)
Region Coverage: 28.12% (35155/125013)
Branch Coverage: 24.91% (17965/72126)
Coverage Report: http://coverage.selectdb-in.cc/coverage/250aad0379597a0309c9d0d87a54a98f92a069b6_250aad0379597a0309c9d0d87a54a98f92a069b6/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1381926031
##########
be/src/vec/columns/predicate_column.h:
##########
@@ -220,29 +201,60 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<
}
void insert_many_date(const char* data_ptr, size_t num) {
- size_t intput_type_size = sizeof(uint24_t);
- size_t res_type_size = sizeof(uint32_t);
+ constexpr size_t input_type_size = sizeof(PrimitiveTypeTraits<TYPE_DATE>::StorageFieldType);
+ static_assert(input_type_size == sizeof(uint24_t));
char* input_data_ptr = const_cast<char*>(data_ptr);
- char* res_ptr = (char*)data.get_end_ptr();
- memset(res_ptr, 0, res_type_size * num);
+ auto* res_ptr = reinterpret_cast<VecDateTimeValue*>(data.get_end_ptr());
for (int i = 0; i < num; i++) {
- memcpy(res_ptr, input_data_ptr, intput_type_size);
- res_ptr += res_type_size;
- input_data_ptr += intput_type_size;
+ uint64_t value = *reinterpret_cast<uint24_t*>(input_data_ptr);
+ res_ptr[i].set_olap_date(value);
+ input_data_ptr += input_type_size;
+ }
+ data.set_end_ptr(res_ptr + num);
+ }
+
+ void insert_many_datetime(const char* data_ptr, size_t num) {
+ constexpr size_t input_type_size =
+ sizeof(PrimitiveTypeTraits<TYPE_DATETIME>::StorageFieldType);
+ static_assert(input_type_size == sizeof(uint64_t));
+ char* input_data_ptr = const_cast<char*>(data_ptr);
+
+ auto* res_ptr = reinterpret_cast<VecDateTimeValue*>(data.get_end_ptr());
+ for (int i = 0; i < num; i++) {
+ res_ptr[i].from_olap_datetime(*reinterpret_cast<uint64_t*>(input_data_ptr));
+ input_data_ptr += input_type_size;
+ }
+ data.set_end_ptr(res_ptr + num);
+ }
+
+ // The logic is same to ColumnDecimal::insert_many_fix_len_data
+ void insert_many_decimalv2(const char* data_ptr, size_t num) {
+ size_t old_size = data.size();
+ data.resize(old_size + num);
+
+ DecimalV2Value* target = (DecimalV2Value*)(data.data() + old_size);
+ for (int i = 0; i < num; i++) {
+ const char* cur_ptr = data_ptr + sizeof(decimal12_t) * i;
+ int64_t int_value = unaligned_load<int64_t>(cur_ptr);
+ int32_t frac_value = *(int32_t*)(cur_ptr + sizeof(int64_t));
+ target[i].from_olap_decimal(int_value, frac_value);
}
- data.set_end_ptr(res_ptr);
}
void insert_many_fix_len_data(const char* data_ptr, size_t num) override {
- if constexpr (std::is_same_v<T, decimal12_t>) {
Review Comment:
For normal primitive types(e.g. TYPE_INT/TYPE_BIGINT/TYPE_DECIMAL32/TYPE_DECIMAL64), there is no need to convert between compute layer type and storage layer type.
Therefore, we can directly use the `memcpy` function to perform bulk data copying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790514572
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1791682207
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809411968
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.5 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162452704 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378863887
##########
be/src/olap/in_list_predicate.h:
##########
@@ -134,9 +121,8 @@ class InListPredicateBase : public ColumnPredicate {
} else if constexpr (Type == TYPE_DECIMALV2) {
HybridSetBase::IteratorBase* iter = hybrid_set->begin();
while (iter->has_next()) {
- const DecimalV2Value* value = (const DecimalV2Value*)(iter->get_value());
- decimal12_t decimal12 = {value->int_value(), value->frac_value()};
- _values->insert(&decimal12);
+ const auto* value = (const DecimalV2Value*)(iter->get_value());
+ _values->insert(value);
iter->next();
}
} else if constexpr (Type == TYPE_DATE) {
Review Comment:
should also modify datev1 and datetimev1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1378875692
##########
be/src/olap/comparison_predicate.h:
##########
@@ -107,8 +120,22 @@ class ComparisonPredicateBase : public ColumnPredicate {
}
roaring::Roaring roaring;
- RETURN_IF_ERROR(iterator->read_from_inverted_index(column_name, &_value, query_type,
- num_rows, &roaring));
+ if constexpr (Type == PrimitiveType::TYPE_DATE) {
+ uint24_t date = _value.to_olap_date();
Review Comment:
Maybe we can uniformly implement a function for converting CppType and StorageType to each other.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812317721
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.61 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.3 seconds inserted 10000000 Rows, about 353K ops/s
storage size: 17097722596 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1808023678
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit f0b58300133758d0bdccf41444ffe7059042ef81, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5308 5100 5155 5100
q2 358 161 154 154
q3 2046 1988 2059 1988
q4 1414 1373 1362 1362
q5 3968 3931 4019 3931
q6 245 129 131 129
q7 1457 879 907 879
q8 2790 2790 2779 2779
q9 9839 9528 9393 9393
q10 3479 3551 3540 3540
q11 390 252 257 252
q12 452 299 300 299
q13 4560 4129 4148 4129
q14 320 292 288 288
q15 653 543 553 543
q16 693 590 592 590
q17 1140 1076 1072 1072
q18 8073 7575 7653 7575
q19 1686 1684 1709 1684
q20 579 338 300 300
q21 4701 4382 4366 4366
q22 510 423 410 410
Total cold run time: 54661 ms
Total hot run time: 50763 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5036 5227 5071 5071
q2 370 241 263 241
q3 4073 4076 4082 4076
q4 2831 2772 2787 2772
q5 9731 9696 9631 9631
q6 248 123 123 123
q7 3077 2506 2526 2506
q8 4833 4819 4818 4818
q9 13095 12942 12898 12898
q10 4108 4202 4203 4202
q11 810 683 660 660
q12 1011 851 850 850
q13 4286 3846 3882 3846
q14 387 337 360 337
q15 625 556 547 547
q16 760 706 683 683
q17 3909 3999 3947 3947
q18 9647 9403 9543 9403
q19 1884 1769 1791 1769
q20 2422 2089 2065 2065
q21 9069 9013 8824 8824
q22 920 845 854 845
Total cold run time: 83132 ms
Total hot run time: 80114 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812476358
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1788309894
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #26191:
URL: https://github.com/apache/doris/pull/26191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812355406
TeamCity be ut coverage result:
Function Coverage: 36.80% (8418/22875)
Line Coverage: 29.29% (68411/233560)
Region Coverage: 27.88% (35353/126810)
Branch Coverage: 24.65% (18071/73312)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6dbc50f5131629ffa647a433564f4c7597afc292_6dbc50f5131629ffa647a433564f4c7597afc292/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790482597
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1801023714
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.14 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17161946349 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1386189902
##########
be/src/exprs/bloom_filter_func.h:
##########
@@ -416,37 +416,10 @@ struct FixedStringFindOp : public StringFindOp {
}
};
-struct DateTimeFindOp : public CommonFindOp<VecDateTimeValue> {
- bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- VecDateTimeValue value;
- value.from_olap_datetime(*reinterpret_cast<const uint64_t*>(data));
- return bloom_filter.test(Slice((char*)&value, sizeof(VecDateTimeValue)));
- }
-};
-
-// avoid violating C/C++ aliasing rules.
-// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101684
-
-struct DateFindOp : public CommonFindOp<VecDateTimeValue> {
- bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- uint24_t date = *static_cast<const uint24_t*>(data);
- uint64_t value = uint32_t(date);
-
- VecDateTimeValue date_value;
- date_value.from_olap_date(value);
-
- return bloom_filter.test(Slice((char*)&date_value, sizeof(VecDateTimeValue)));
- }
-};
-
struct DecimalV2FindOp : public CommonFindOp<DecimalV2Value> {
bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void* data) const {
- auto packed_decimal = *static_cast<const decimal12_t*>(data);
- DecimalV2Value value;
- int64_t int_value = packed_decimal.integer;
- int32_t frac_value = packed_decimal.fraction;
- value.from_olap_decimal(int_value, frac_value);
-
+ // Predicate column using decimalv2value as column value type, so not need convert here
+ DecimalV2Value value = *static_cast<const DecimalV2Value*>(data);
Review Comment:
Could we delete DecimalV2FindOp??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1791729348
TeamCity be ut coverage result:
Function Coverage: 37.28% (8453/22673)
Line Coverage: 29.66% (68405/230619)
Region Coverage: 28.34% (35414/124983)
Branch Coverage: 25.07% (18069/72076)
Coverage Report: http://coverage.selectdb-in.cc/coverage/741fb6fde2b24deb28ad8fe75457b1c7410e2147_741fb6fde2b24deb28ad8fe75457b1c7410e2147/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1381320218
##########
be/src/vec/columns/predicate_column.h:
##########
@@ -220,29 +201,60 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<
}
void insert_many_date(const char* data_ptr, size_t num) {
- size_t intput_type_size = sizeof(uint24_t);
- size_t res_type_size = sizeof(uint32_t);
+ constexpr size_t input_type_size = sizeof(PrimitiveTypeTraits<TYPE_DATE>::StorageFieldType);
+ static_assert(input_type_size == sizeof(uint24_t));
char* input_data_ptr = const_cast<char*>(data_ptr);
- char* res_ptr = (char*)data.get_end_ptr();
- memset(res_ptr, 0, res_type_size * num);
+ auto* res_ptr = reinterpret_cast<VecDateTimeValue*>(data.get_end_ptr());
for (int i = 0; i < num; i++) {
- memcpy(res_ptr, input_data_ptr, intput_type_size);
- res_ptr += res_type_size;
- input_data_ptr += intput_type_size;
+ uint64_t value = *reinterpret_cast<uint24_t*>(input_data_ptr);
+ res_ptr[i].set_olap_date(value);
+ input_data_ptr += input_type_size;
+ }
+ data.set_end_ptr(res_ptr + num);
+ }
+
+ void insert_many_datetime(const char* data_ptr, size_t num) {
+ constexpr size_t input_type_size =
+ sizeof(PrimitiveTypeTraits<TYPE_DATETIME>::StorageFieldType);
+ static_assert(input_type_size == sizeof(uint64_t));
+ char* input_data_ptr = const_cast<char*>(data_ptr);
+
+ auto* res_ptr = reinterpret_cast<VecDateTimeValue*>(data.get_end_ptr());
+ for (int i = 0; i < num; i++) {
+ res_ptr[i].from_olap_datetime(*reinterpret_cast<uint64_t*>(input_data_ptr));
+ input_data_ptr += input_type_size;
+ }
+ data.set_end_ptr(res_ptr + num);
+ }
+
+ // The logic is same to ColumnDecimal::insert_many_fix_len_data
+ void insert_many_decimalv2(const char* data_ptr, size_t num) {
+ size_t old_size = data.size();
+ data.resize(old_size + num);
+
+ DecimalV2Value* target = (DecimalV2Value*)(data.data() + old_size);
+ for (int i = 0; i < num; i++) {
+ const char* cur_ptr = data_ptr + sizeof(decimal12_t) * i;
+ int64_t int_value = unaligned_load<int64_t>(cur_ptr);
+ int32_t frac_value = *(int32_t*)(cur_ptr + sizeof(int64_t));
+ target[i].from_olap_decimal(int_value, frac_value);
}
- data.set_end_ptr(res_ptr);
}
void insert_many_fix_len_data(const char* data_ptr, size_t num) override {
- if constexpr (std::is_same_v<T, decimal12_t>) {
Review Comment:
Add a method to convert from StorageFieldType to CppType, and then we can remove the if else in get_zone_map_value/insert_many_fix_len_data, and all methods like insert_many_date/insert_many_decimalv2...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812425740
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 6dbc50f5131629ffa647a433564f4c7597afc292, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5246 5039 5029 5029
q2 366 156 160 156
q3 2029 1921 1921 1921
q4 1375 1265 1244 1244
q5 3930 3900 3987 3900
q6 250 133 131 131
q7 1407 883 871 871
q8 2725 2787 2757 2757
q9 13738 9946 9614 9614
q10 3441 3519 3519 3519
q11 396 258 248 248
q12 447 294 290 290
q13 4599 3827 3776 3776
q14 318 288 277 277
q15 628 564 566 564
q16 655 585 580 580
q17 1119 954 899 899
q18 7880 7409 7354 7354
q19 1684 1678 1672 1672
q20 574 297 294 294
q21 4449 3949 4024 3949
q22 517 417 404 404
Total cold run time: 57773 ms
Total hot run time: 49449 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5051 5093 5008 5008
q2 326 221 225 221
q3 4061 3865 3866 3865
q4 2798 2755 2742 2742
q5 9562 9532 9473 9473
q6 243 127 128 127
q7 2682 2281 2329 2281
q8 4852 4861 4837 4837
q9 13261 13058 13101 13058
q10 4061 4175 4166 4166
q11 754 657 660 657
q12 1008 838 828 828
q13 4276 3654 3561 3561
q14 366 348 345 345
q15 580 563 550 550
q16 755 657 695 657
q17 3855 3862 3949 3862
q18 9457 8901 8996 8901
q19 1819 1755 1785 1755
q20 2406 2048 2038 2038
q21 8706 8460 8601 8460
q22 928 802 853 802
Total cold run time: 81807 ms
Total hot run time: 78194 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812436602
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.55 seconds
stream load tsv: 582 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17098197739 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1800870179
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1808018311
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.84 seconds
stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162250011 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #26191:
URL: https://github.com/apache/doris/pull/26191#discussion_r1392558588
##########
be/src/olap/in_list_predicate.h:
##########
@@ -363,16 +319,16 @@ class InListPredicateBase : public ColumnPredicate {
}
if constexpr (PT == PredicateType::NOT_IN_LIST) {
if constexpr (Type == TYPE_DATE) {
- T tmp_min_uint32_value = 0;
Review Comment:
if constexpr (PT == PredicateType::IN_LIST) {
why not using the same way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812085521
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812169591
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812185784
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1812004440
TeamCity be ut coverage result:
Function Coverage: 36.80% (8420/22879)
Line Coverage: 29.29% (68403/233518)
Region Coverage: 27.88% (35346/126786)
Branch Coverage: 24.64% (18062/73292)
Coverage Report: http://coverage.selectdb-in.cc/coverage/36a7b5f5ad50854548e52cc4b35d34bc0e08edc8_36a7b5f5ad50854548e52cc4b35d34bc0e08edc8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1800891263
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1800879306
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1803025956
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809403600
TeamCity be ut coverage result:
Function Coverage: 36.75% (8407/22875)
Line Coverage: 29.29% (68152/232704)
Region Coverage: 27.85% (35182/126325)
Branch Coverage: 24.66% (17974/72894)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b8f25b740c1c70e22e69804be79b20b0142f6ae8_b8f25b740c1c70e22e69804be79b20b0142f6ae8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1807955280
TeamCity be ut coverage result:
Function Coverage: 36.76% (8408/22873)
Line Coverage: 29.30% (68163/232654)
Region Coverage: 27.86% (35182/126298)
Branch Coverage: 24.67% (17978/72868)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f0b58300133758d0bdccf41444ffe7059042ef81_f0b58300133758d0bdccf41444ffe7059042ef81/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1809382815
TeamCity be ut coverage result:
Function Coverage: 36.75% (8406/22875)
Line Coverage: 29.28% (68142/232694)
Region Coverage: 27.85% (35177/126316)
Branch Coverage: 24.66% (17976/72890)
Coverage Report: http://coverage.selectdb-in.cc/coverage/aac0208bf8be26d516eaaecd2d657012e59a2a68_aac0208bf8be26d516eaaecd2d657012e59a2a68/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1807806077
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1788559952
TeamCity be ut coverage result:
Function Coverage: 37.34% (8470/22682)
Line Coverage: 29.66% (68561/231177)
Region Coverage: 28.23% (35608/126152)
Branch Coverage: 25.09% (18224/72622)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f088cab0b831ec8556731e9e920d468d381d00a7_f088cab0b831ec8556731e9e920d468d381d00a7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26191:
URL: https://github.com/apache/doris/pull/26191#issuecomment-1790647437
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.57 seconds
stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161970149 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org