You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "jacktengg (via GitHub)" <gi...@apache.org> on 2023/12/11 18:56:32 UTC
[PR] [fix](decimal) fix decimal overflow caused by null value [doris]
jacktengg opened a new pull request, #28260:
URL: https://github.com/apache/doris/pull/28260
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1853179286
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851463213
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1850695515
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851186032
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851216825
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.01 seconds
stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 27.7 seconds inserted 10000000 Rows, about 361K ops/s
storage size: 17224048992 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28260:
URL: https://github.com/apache/doris/pull/28260#discussion_r1423005028
##########
be/src/vec/columns/column_nullable.cpp:
##########
@@ -47,9 +48,24 @@ ColumnNullable::ColumnNullable(MutableColumnPtr&& nested_column_, MutableColumnP
if (is_column_const(*null_map)) {
LOG(FATAL) << "ColumnNullable cannot have constant null map";
}
+
+ if (update_null_data_) {
+ update_null_data();
+ }
+
_need_update_has_null = true;
}
+void ColumnNullable::update_null_data() {
Review Comment:
warning: method 'update_null_data' can be made const [readability-make-member-function-const]
```suggestion
void ColumnNullable::update_null_data() const {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1850872726
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.53 seconds
stream load tsv: 590 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17219664113 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851285138
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.43 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17219821600 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1850855765
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 291dbd03bce67feccb7c21ebcfb07fd9dc16a7f1, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4673 4479 4508 4479
q2 369 145 158 145
q3 1457 1267 1249 1249
q4 1129 946 906 906
q5 3150 3156 3173 3156
q6 250 128 128 128
q7 1021 484 478 478
q8 2217 2214 2214 2214
q9 6667 6692 6647 6647
q10 3217 3248 3247 3247
q11 330 211 210 210
q12 350 210 210 210
q13 4698 3860 4018 3860
q14 246 213 210 210
q15 563 522 518 518
q16 453 403 390 390
q17 1010 615 529 529
q18 7761 7428 6999 6999
q19 1512 1349 1448 1349
q20 1796 322 309 309
q21 3105 2675 2685 2675
q22 347 285 288 285
Total cold run time: 46321 ms
Total hot run time: 40193 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4404 4425 4418 4418
q2 268 168 173 168
q3 3557 3552 3540 3540
q4 2381 2384 2385 2384
q5 5761 5743 5750 5743
q6 237 121 123 121
q7 2388 1863 1861 1861
q8 3525 3510 3520 3510
q9 9068 9103 9071 9071
q10 3910 3965 3992 3965
q11 510 425 416 416
q12 769 601 596 596
q13 4333 3545 3570 3545
q14 293 265 259 259
q15 572 521 516 516
q16 520 440 505 440
q17 1860 1846 1870 1846
q18 8747 8727 8379 8379
q19 1740 1731 1760 1731
q20 2241 1968 1932 1932
q21 6547 6228 6185 6185
q22 536 446 417 417
Total cold run time: 64167 ms
Total hot run time: 61043 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851665648
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit d9fb9e79d51e0832a492eb4d1bc9945aa73348ff, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4706 4457 4467 4457
q2 357 158 159 158
q3 1473 1259 1237 1237
q4 1107 882 871 871
q5 3188 3213 3190 3190
q6 247 128 128 128
q7 1030 489 481 481
q8 2208 2269 2213 2213
q9 6715 6757 6662 6662
q10 3281 3261 3276 3261
q11 331 194 205 194
q12 354 203 208 203
q13 6790 3786 3792 3786
q14 249 213 214 213
q15 580 527 527 527
q16 447 395 396 395
q17 1011 606 528 528
q18 7492 7144 6839 6839
q19 1539 1533 1329 1329
q20 533 331 289 289
q21 3101 2638 2651 2638
q22 352 274 283 274
Total cold run time: 47091 ms
Total hot run time: 39873 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4421 4386 4382 4382
q2 268 164 171 164
q3 3512 3486 3491 3486
q4 2403 2390 2390 2390
q5 5727 5729 5738 5729
q6 235 122 121 121
q7 2358 1870 1885 1870
q8 3526 3519 3525 3519
q9 9068 9042 8979 8979
q10 3933 3993 4002 3993
q11 509 376 390 376
q12 762 617 618 617
q13 4325 3552 3530 3530
q14 285 251 249 249
q15 566 535 515 515
q16 493 465 489 465
q17 1879 1867 1868 1867
q18 8498 8167 8068 8068
q19 1722 1773 1774 1773
q20 2265 1954 1929 1929
q21 6529 6152 6125 6125
q22 493 418 413 413
Total cold run time: 63777 ms
Total hot run time: 60560 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1853179253
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #28260:
URL: https://github.com/apache/doris/pull/28260
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851275939
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 80aaf2bbc13d0513a23a13a00da50ea189113d34, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4650 4453 4434 4434
q2 366 152 157 152
q3 1447 1196 1261 1196
q4 1120 905 877 877
q5 3116 3135 3106 3106
q6 248 125 124 124
q7 984 499 476 476
q8 2181 2221 2177 2177
q9 6665 6606 6659 6606
q10 3230 3268 3244 3244
q11 333 206 201 201
q12 347 207 207 207
q13 4554 3807 3832 3807
q14 237 212 207 207
q15 565 526 522 522
q16 445 385 399 385
q17 1011 563 532 532
q18 7665 7418 7084 7084
q19 1519 1320 1393 1320
q20 507 314 292 292
q21 3026 2691 2677 2677
q22 352 281 288 281
Total cold run time: 44568 ms
Total hot run time: 39907 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4410 4389 4413 4389
q2 266 162 169 162
q3 3543 3545 3525 3525
q4 2380 2370 2368 2368
q5 5745 5744 5733 5733
q6 240 120 121 120
q7 2392 1886 1860 1860
q8 3508 3523 3513 3513
q9 9054 9027 9025 9025
q10 3898 3984 3997 3984
q11 494 388 387 387
q12 766 585 586 585
q13 4298 3579 3574 3574
q14 292 254 252 252
q15 569 518 517 517
q16 516 475 475 475
q17 1872 1879 1862 1862
q18 8698 8379 8444 8379
q19 1705 1743 1740 1740
q20 2278 2046 1938 1938
q21 6530 6180 6148 6148
q22 515 435 419 419
Total cold run time: 63969 ms
Total hot run time: 60955 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851208676
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 2d1ef66886cf5762493775cd6b08c0c34c8300b9, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4618 4458 4500 4458
q2 384 169 162 162
q3 1452 1198 1232 1198
q4 1097 912 866 866
q5 3138 3106 3100 3100
q6 250 130 123 123
q7 976 489 474 474
q8 2171 2251 2188 2188
q9 6637 6680 6661 6661
q10 3181 3242 3257 3242
q11 328 212 210 210
q12 347 201 200 200
q13 4537 5562 3981 3981
q14 262 217 214 214
q15 562 528 512 512
q16 451 384 383 383
q17 998 594 546 546
q18 7469 8322 9242 8322
q19 1496 1297 1452 1297
q20 540 340 437 340
q21 3033 2656 2653 2653
q22 349 279 288 279
Total cold run time: 44276 ms
Total hot run time: 41409 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4387 4355 4397 4355
q2 272 165 171 165
q3 3558 3525 3529 3525
q4 2367 2366 2362 2362
q5 5736 5757 5749 5749
q6 244 120 121 120
q7 2379 1867 1888 1867
q8 3515 3510 3517 3510
q9 9088 9051 9005 9005
q10 3893 3991 3995 3991
q11 498 393 373 373
q12 767 591 627 591
q13 4282 3560 3565 3560
q14 285 256 257 256
q15 571 521 524 521
q16 498 473 470 470
q17 1856 1862 1821 1821
q18 8674 8289 8570 8289
q19 1745 1728 1729 1728
q20 2245 1941 1950 1941
q21 6508 6198 6173 6173
q22 493 415 434 415
Total cold run time: 63861 ms
Total hot run time: 60787 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851252910
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](decimal) fix decimal overflow caused by null value [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28260:
URL: https://github.com/apache/doris/pull/28260#issuecomment-1851669351
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.73 seconds
stream load tsv: 586 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17217033626 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org