You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2023/12/14 07:06:54 UTC
[PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Mryange opened a new pull request, #28395:
URL: https://github.com/apache/doris/pull/28395
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1857341703
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1857896568
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1857415785
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.89 seconds
stream load tsv: 589 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17219829154 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1857228468
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1855443560
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 7d25f17d7d23589b7cfbfc324855a5e4ca566914, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4691 4452 4463 4452
q2 363 151 158 151
q3 1466 1268 1222 1222
q4 1114 917 859 859
q5 3153 3138 3130 3130
q6 247 127 127 127
q7 989 489 484 484
q8 2212 2206 2181 2181
q9 6670 6647 6659 6647
q10 3207 3237 3253 3237
q11 321 209 199 199
q12 345 198 193 193
q13 4567 3802 3788 3788
q14 241 211 216 211
q15 566 530 526 526
q16 445 379 378 378
q17 1009 588 582 582
q18 7184 6919 6957 6919
q19 1507 1363 1385 1363
q20 536 294 305 294
q21 3079 2638 2632 2632
q22 347 279 281 279
Total cold run time: 44259 ms
Total hot run time: 39854 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4393 4382 4391 4382
q2 264 163 174 163
q3 3523 3523 3505 3505
q4 2369 2351 2354 2351
q5 5713 5720 5718 5718
q6 241 119 121 119
q7 2374 1880 1890 1880
q8 3523 3533 3524 3524
q9 9047 9029 8982 8982
q10 3912 3967 3995 3967
q11 497 370 393 370
q12 759 613 589 589
q13 4293 3549 3530 3530
q14 296 251 257 251
q15 574 527 524 524
q16 497 469 467 467
q17 1881 1848 1834 1834
q18 8759 8289 8209 8209
q19 1733 1729 1719 1719
q20 2247 1937 1939 1937
q21 6516 6162 6173 6162
q22 508 406 444 406
Total cold run time: 63919 ms
Total hot run time: 60589 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange closed pull request #28395: [feature](expr) Add length limit to the debug string of expr.
URL: https://github.com/apache/doris/pull/28395
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28395:
URL: https://github.com/apache/doris/pull/28395#discussion_r1426268241
##########
be/src/vec/exprs/vexpr.cpp:
##########
@@ -444,19 +444,31 @@ std::string VExpr::debug_string() const {
out << " children=" << debug_string(_children);
}
- return out.str();
+ return limit_debug_string(out);
+}
+
+bool VExpr::check_string_within_limit(std::stringstream& out) {
Review Comment:
warning: method 'check_string_within_limit' can be made static [readability-convert-member-functions-to-static]
```suggestion
static bool VExpr::check_string_within_limit(std::stringstream& out) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1855445036
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.19 seconds
stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17219866941 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1858032703
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 0cc420fbc2747d0ae5c286e0cbaefef4e6b839cd, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4787 4506 4543 4506
q2 361 154 158 154
q3 1478 1297 1232 1232
q4 1125 919 920 919
q5 3190 3198 3184 3184
q6 247 128 129 128
q7 996 491 483 483
q8 2235 2256 2243 2243
q9 6728 6719 6681 6681
q10 3242 3302 3282 3282
q11 325 206 199 199
q12 357 206 207 206
q13 4562 3802 3767 3767
q14 244 207 211 207
q15 572 523 522 522
q16 444 402 383 383
q17 1035 580 580 580
q18 7133 6729 6836 6729
q19 1566 1464 1352 1352
q20 514 340 290 290
q21 3072 2654 2656 2654
q22 352 279 283 279
Total cold run time: 44565 ms
Total hot run time: 39980 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4466 4466 4453 4453
q2 270 164 171 164
q3 3509 3512 3506 3506
q4 2393 2373 2384 2373
q5 5734 5775 5743 5743
q6 243 121 123 121
q7 2373 1838 1843 1838
q8 3554 3557 3555 3555
q9 9021 8999 8968 8968
q10 3949 4025 4019 4019
q11 504 378 373 373
q12 764 591 593 591
q13 4274 3554 3569 3554
q14 290 257 261 257
q15 582 516 528 516
q16 532 453 455 453
q17 1862 1858 1857 1857
q18 8468 8175 8660 8175
q19 1737 1777 1775 1775
q20 2247 1966 1924 1924
q21 6551 6193 6167 6167
q22 528 412 420 412
Total cold run time: 63851 ms
Total hot run time: 60794 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1913121412
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1913143965
TeamCity be ut coverage result:
Function Coverage: 36.18% (8621/23827)
Line Coverage: 28.26% (70516/249565)
Region Coverage: 27.25% (36386/133544)
Branch Coverage: 24.05% (18648/77550)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ac14ec1b85cb742b4a9945438f1009fbca952ccf_ac14ec1b85cb742b4a9945438f1009fbca952ccf/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #28395:
URL: https://github.com/apache/doris/pull/28395#discussion_r1426922885
##########
be/src/vec/exprs/vcast_expr.cpp:
##########
@@ -142,8 +142,11 @@ std::string VCastExpr::debug_string() const {
out << ",";
}
out << input_expr->debug_string();
Review Comment:
这么改感觉像是每个expr < 4096,但是如果是多个expr 还是会崩溃。
我认为，我们应该给debug_string 传递一个(stringstream) 然后判断啊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1855262685
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28395:
URL: https://github.com/apache/doris/pull/28395#discussion_r1427522692
##########
be/src/vec/exprs/vcase_expr.cpp:
##########
@@ -117,20 +117,21 @@ const std::string& VCaseExpr::expr_name() const {
return _expr_name;
}
-std::string VCaseExpr::debug_string() const {
- std::stringstream out;
+void VCaseExpr::debug_string(std::stringstream& out) const {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VCaseExpr::debug_string(std::stringstream& out) {
```
##########
be/src/vec/exprs/vcast_expr.cpp:
##########
@@ -130,20 +130,21 @@ const std::string& VCastExpr::expr_name() const {
return _expr_name;
}
-std::string VCastExpr::debug_string() const {
- std::stringstream out;
+void VCastExpr::debug_string(std::stringstream& out) const {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VCastExpr::debug_string(std::stringstream& out) {
```
##########
be/src/vec/exprs/vexpr.cpp:
##########
@@ -467,6 +486,16 @@
return debug_string(exprs);
}
+void VExpr::debug_string(const VExprSPtrs& exprs, std::stringstream& out) {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VExpr::debug_string(const VExprSPtrs& exprs, std::stringstream& out) {
```
##########
be/src/vec/exprs/vlambda_function_call_expr.h:
##########
@@ -57,8 +57,10 @@ class VLambdaFunctionCallExpr : public VExpr {
return _lambda_function->execute(context, block, result_column_id, _data_type, _children);
}
- std::string debug_string() const override {
- std::stringstream out;
+ void debug_string(std::stringstream& out) const override {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void debug_string(std::stringstream& out) override {
```
##########
be/src/vec/exprs/vmatch_predicate.cpp:
##########
@@ -146,17 +146,21 @@ const std::string& VMatchPredicate::function_name() const {
return _function_name;
}
-std::string VMatchPredicate::debug_string() const {
- std::stringstream out;
- out << "MatchPredicate(" << children()[0]->debug_string() << ",[";
+void VMatchPredicate::debug_string(std::stringstream& out) const {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VMatchPredicate::debug_string(std::stringstream& out) {
```
##########
be/src/vec/exprs/vectorized_fn_call.cpp:
##########
@@ -198,8 +198,10 @@ const std::string& VectorizedFnCall::expr_name() const {
return _expr_name;
}
-std::string VectorizedFnCall::debug_string() const {
- std::stringstream out;
+void VectorizedFnCall::debug_string(std::stringstream& out) const {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VectorizedFnCall::debug_string(std::stringstream& out) {
```
##########
be/src/vec/exprs/vexpr.cpp:
##########
@@ -438,13 +438,32 @@
std::string VExpr::debug_string() const {
// TODO: implement partial debug string for member vars
std::stringstream out;
+ debug_string(out);
+ return std::move(out).str();
+}
+
+void VExpr::debug_string(std::stringstream& out) const {
+ if (check_string_over_limit(out)) {
+ return;
+ }
out << " type=" << _type.debug_string();
if (!_children.empty()) {
- out << " children=" << debug_string(_children);
+ out << " children=";
+ debug_string(_children, out);
}
+}
- return out.str();
+bool VExpr::check_string_over_limit(std::stringstream& out) {
Review Comment:
warning: method 'check_string_over_limit' can be made static [readability-convert-member-functions-to-static]
```suggestion
static bool VExpr::check_string_over_limit(std::stringstream& out) {
```
##########
be/src/vec/exprs/vexpr.cpp:
##########
@@ -438,13 +438,32 @@ Status VExpr::clone_if_not_exists(const VExprContextSPtrs& ctxs, RuntimeState* s
std::string VExpr::debug_string() const {
// TODO: implement partial debug string for member vars
std::stringstream out;
+ debug_string(out);
+ return std::move(out).str();
+}
+
+void VExpr::debug_string(std::stringstream& out) const {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void VExpr::debug_string(std::stringstream& out) {
```
##########
be/src/vec/exprs/vcolumn_ref.h:
##########
@@ -57,11 +57,13 @@ class VColumnRef final : public VExpr {
const std::string& expr_name() const override { return _column_name; }
- std::string debug_string() const override {
- std::stringstream out;
- out << "VColumnRef(slot_id: " << _column_id << ",column_name: " << _column_name
- << VExpr::debug_string() << ")";
- return out.str();
+ void debug_string(std::stringstream& out) const override {
Review Comment:
warning: method 'debug_string' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void debug_string(std::stringstream& out) override {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](expr) Add length limit to the debug string of expr. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28395:
URL: https://github.com/apache/doris/pull/28395#issuecomment-1858051103
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.41 seconds
stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17222609315 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org