You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yulei-Yang (via GitHub)" <gi...@apache.org> on 2023/12/07 14:19:11 UTC
[PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Yulei-Yang opened a new pull request, #28145:
URL: https://github.com/apache/doris/pull/28145
show_create_database if designed for internal database, it's not suitable for hms catalog
Before:
<img width="873" alt="image" src="https://github.com/apache/doris/assets/28004179/d1ac1e12-97ab-4230-8172-a5c7d5d97be4">
After(same like result in hive):
Already tested in hive v2.1.1, v2.3.9, v3.1.3
If used in a catalog neither internal or hms catalog:
<img width="1232" alt="image" src="https://github.com/apache/doris/assets/28004179/78119908-f845-4cc6-887f-7bb0fc7d489a">
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "Yulei-Yang (via GitHub)" <gi...@apache.org>.
Yulei-Yang commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846475664
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1845494545
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.28 seconds
stream load tsv: 589 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17214715819 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846591014
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.39 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17214803035 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "Yulei-Yang (via GitHub)" <gi...@apache.org>.
Yulei-Yang commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846504016
run clickbench
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #28145:
URL: https://github.com/apache/doris/pull/28145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "Yulei-Yang (via GitHub)" <gi...@apache.org>.
Yulei-Yang commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846564222
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846514528
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846514556
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "Yulei-Yang (via GitHub)" <gi...@apache.org>.
Yulei-Yang commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846565068
run clickbench
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1846593235
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 19e813a0d8a52196017583eee1a65d85a6a38a5b, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4685 4442 4480 4442
q2 371 142 158 142
q3 1466 1271 1239 1239
q4 1110 952 887 887
q5 3117 3135 3159 3135
q6 252 133 127 127
q7 991 485 477 477
q8 2205 2236 2156 2156
q9 6644 6669 6683 6669
q10 3229 3266 3248 3248
q11 329 203 200 200
q12 354 207 210 207
q13 5599 3773 3848 3773
q14 239 208 223 208
q15 565 523 516 516
q16 439 393 393 393
q17 1007 609 585 585
q18 7468 7020 8935 7020
q19 1514 1407 1415 1407
q20 511 309 298 298
q21 3054 2648 2704 2648
q22 348 278 282 278
Total cold run time: 45497 ms
Total hot run time: 40055 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4427 4467 4448 4448
q2 268 161 178 161
q3 3547 3533 3529 3529
q4 2373 2376 2366 2366
q5 5728 5724 5726 5724
q6 241 121 122 121
q7 2374 1857 1889 1857
q8 3524 3518 3538 3518
q9 9027 8978 9025 8978
q10 3888 3991 3994 3991
q11 514 380 381 380
q12 763 606 605 605
q13 4324 3584 3606 3584
q14 291 251 253 251
q15 605 518 515 515
q16 513 442 462 442
q17 1903 1871 1863 1863
q18 8585 8265 8321 8265
q19 1743 1729 1779 1729
q20 2245 1950 1929 1929
q21 6506 6218 6117 6117
q22 508 411 433 411
Total cold run time: 63897 ms
Total hot run time: 60784 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1845476340
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit c988b6716ba4cdb6b1848d7bc8871a5d5c8360f4, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4687 4472 4461 4461
q2 368 145 158 145
q3 1443 1230 1246 1230
q4 1117 922 907 907
q5 3138 3140 3141 3140
q6 252 132 127 127
q7 995 481 469 469
q8 2187 2226 2166 2166
q9 6664 6717 6678 6678
q10 3231 3276 3270 3270
q11 325 214 207 207
q12 346 201 204 201
q13 4617 3832 3798 3798
q14 247 207 210 207
q15 569 521 519 519
q16 433 383 380 380
q17 1020 603 564 564
q18 7451 6984 8812 6984
q19 1506 1384 1397 1384
q20 545 288 319 288
q21 3087 2609 2690 2609
q22 351 284 279 279
Total cold run time: 44579 ms
Total hot run time: 40013 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4365 4394 4379 4379
q2 263 162 170 162
q3 3527 3506 3497 3497
q4 2372 2362 2362 2362
q5 5733 5735 5734 5734
q6 240 119 120 119
q7 2376 1877 1903 1877
q8 3507 3509 3525 3509
q9 9006 8969 8933 8933
q10 3916 4001 3980 3980
q11 499 381 387 381
q12 780 603 595 595
q13 4296 3589 3565 3565
q14 282 250 258 250
q15 569 512 515 512
q16 496 452 496 452
q17 1872 1848 1833 1833
q18 8684 8122 8228 8122
q19 1722 1749 1746 1746
q20 2257 1932 1942 1932
q21 6508 6165 6154 6154
q22 490 439 419 419
Total cold run time: 63760 ms
Total hot run time: 60513 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](hms catalog) support show_create_database for hms catalog [doris]
Posted by "Yulei-Yang (via GitHub)" <gi...@apache.org>.
Yulei-Yang commented on PR #28145:
URL: https://github.com/apache/doris/pull/28145#issuecomment-1845428823
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org