You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "allenhooo (via GitHub)" <gi...@apache.org> on 2024/01/23 03:48:09 UTC
[PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
allenhooo opened a new pull request, #30256:
URL: https://github.com/apache/doris/pull/30256
[fix](common) If the properties in DDL is not provided, the mysql client will lost connection, and no error message is output on the client side.
## Case
If the properties in DDL is not provided (wrong DDL), the mysql client will lost connection:
```sql
mysql> CREATE EXTERNAL RESOURCE 'mysql_v';
ERROR 2013 (HY000): Lost connection to MySQL server during query
```
The correct error message should appear here.
Here is the stack trace:
’‘’
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Resource properties can't be null
at org.apache.doris.analysis.CreateResourceStmt.analyze(CreateResourceStmt.java:84) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:1080) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:728) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:509) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:462) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:245) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:166) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:193) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:246) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]
2024-01-23 09:31:51,757 WARN (mysql-nio-pool-1|316) [ConnectProcessor.handleQueryException():329] Process one query failed because unknown reason:
java.lang.NullPointerException: Cannot invoke "java.util.Map.entrySet()" because "this.map" is null
at org.apache.doris.common.util.PrintableMap.showEntries(PrintableMap.java:108) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.PrintableMap.toString(PrintableMap.java:97) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.lang.String.valueOf(Unknown Source) ~[?:?]
at java.lang.StringBuilder.append(Unknown Source) ~[?:?]
at org.apache.doris.analysis.CreateResourceStmt.toSql(CreateResourceStmt.java:111) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.AuditLogHelper.logAuditLog(AuditLogHelper.java:101) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.auditAfterExec(ConnectProcessor.java:171) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:267) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:166) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:193) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:246) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]
‘’‘
It's because **PrintableMap.toString()** does not correctly handle the situation where map is null.
The test results are as follows：
'''sql
mysql> CREATE EXTERNAL RESOURCE 'mysql_v';
ERROR 1105 (HY000): errCode = 2, detailMessage = Resource properties can't be null
'''
## Further comments
The modification is small, please evaluate the side effects of this modification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1905448648
<details>
<summary>TPC-DS: <b>Total hot run time: 186549 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9aa2bf7c4c7921c1923f9369d78cbd13a60cf145, data reload: false
query1 968 340 329 329
query2 6556 1986 1977 1977
query3 6706 201 197 197
query4 32014 22200 22140 22140
query5 4452 446 360 360
query6 243 153 164 153
query7 4600 271 260 260
query8 226 169 176 169
query9 8347 2529 2531 2529
query10 411 234 232 232
query11 16926 15572 15530 15530
query12 120 71 65 65
query13 1692 386 395 386
query14 10585 6922 7034 6922
query15 215 176 179 176
query16 5799 257 251 251
query17 945 476 469 469
query18 1791 269 251 251
query19 171 138 132 132
query20 70 71 68 68
query21 186 127 128 127
query22 5142 4907 4936 4907
query23 31368 30887 30813 30813
query24 7970 2816 2809 2809
query25 511 315 314 314
query26 883 145 144 144
query27 2653 293 286 286
query28 3883 1848 1849 1848
query29 909 618 599 599
query30 267 140 141 140
query31 936 750 760 750
query32 73 51 49 49
query33 427 212 209 209
query34 822 453 466 453
query35 887 742 777 742
query36 1304 1157 1250 1157
query37 93 59 62 59
query38 3406 3252 3225 3225
query39 1317 1254 1262 1254
query40 191 88 82 82
query41 37 37 34 34
query42 88 83 77 77
query43 519 499 488 488
query44 1045 690 696 690
query45 191 175 172 172
query46 1063 653 657 653
query47 1686 1517 1515 1515
query48 394 304 309 304
query49 937 280 285 280
query50 691 311 311 311
query51 5258 5156 5119 5119
query52 87 84 74 74
query53 322 253 264 253
query54 205 186 185 185
query55 83 72 73 72
query56 176 174 167 167
query57 1023 943 888 888
query58 186 159 154 154
query59 3060 2760 2683 2683
query60 213 190 197 190
query61 85 81 81 81
query62 499 354 359 354
query63 279 249 253 249
query64 3677 1765 1734 1734
query65 3377 3270 3236 3236
query66 1216 315 328 315
query67 15828 15102 14973 14973
query68 12821 542 513 513
query69 591 311 295 295
query70 1637 1454 1478 1454
query71 10435 10203 10202 10202
query72 5037 2841 2822 2822
query73 2984 306 308 306
query74 7139 6354 6422 6354
query75 5191 2288 2255 2255
query76 6435 1097 1028 1028
query77 670 227 233 227
query78 9646 9039 8728 8728
query79 994 500 501 500
query80 587 331 319 319
query81 475 208 200 200
query82 207 79 78 78
query83 144 118 117 117
query84 280 70 71 70
query85 1101 329 328 328
query86 386 336 377 336
query87 3473 3358 3319 3319
query88 2731 2203 2180 2180
query89 451 374 354 354
query90 2119 187 184 184
query91 161 129 124 124
query92 54 46 44 44
query93 1491 459 449 449
query94 1354 158 160 158
query95 509 477 449 449
query96 624 313 315 313
query97 4246 4125 4150 4125
query98 211 192 189 189
query99 975 730 715 715
Total cold run time: 293449 ms
Total hot run time: 186549 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1905459594
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 9aa2bf7c4c7921c1923f9369d78cbd13a60cf145 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.9 seconds inserted 10000000 Rows, about 671K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1905455546
<details>
<summary>ClickBench: <b>Total hot run time: 30.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9aa2bf7c4c7921c1923f9369d78cbd13a60cf145, data reload: false
query1 0.04 0.02 0.02
query2 0.05 0.02 0.03
query3 0.22 0.05 0.05
query4 1.71 0.08 0.07
query5 0.53 0.52 0.52
query6 1.27 0.62 0.63
query7 0.02 0.01 0.01
query8 0.05 0.03 0.02
query9 0.55 0.51 0.50
query10 0.56 0.58 0.55
query11 0.12 0.09 0.09
query12 0.12 0.09 0.09
query13 0.61 0.61 0.60
query14 0.78 0.82 0.82
query15 0.79 0.76 0.76
query16 0.39 0.37 0.39
query17 1.01 1.04 1.03
query18 0.25 0.25 0.25
query19 1.83 1.83 1.80
query20 0.02 0.01 0.01
query21 15.40 0.57 0.57
query22 2.80 2.40 1.63
query23 17.37 0.79 0.89
query24 2.32 1.65 0.84
query25 0.42 0.27 0.24
query26 0.52 0.14 0.14
query27 0.06 0.05 0.06
query28 11.14 0.78 0.77
query29 12.52 3.25 3.25
query30 0.60 0.50 0.49
query31 2.77 0.36 0.36
query32 3.35 0.48 0.50
query33 3.29 3.24 3.24
query34 15.85 4.24 4.31
query35 4.35 4.32 4.28
query36 1.11 1.09 1.08
query37 0.07 0.05 0.06
query38 0.04 0.03 0.03
query39 0.02 0.02 0.01
query40 0.16 0.13 0.14
query41 0.06 0.01 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.18 s
Total hot run time: 30.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #30256:
URL: https://github.com/apache/doris/pull/30256
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "allenhooo (via GitHub)" <gi...@apache.org>.
allenhooo commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1905374725
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1905435390
<details>
<summary>TPC-H: <b>Total hot run time: 38709 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9aa2bf7c4c7921c1923f9369d78cbd13a60cf145, data reload: false
------ Round 1 ----------------------------------
q1 17621 5349 5337 5337
q2 2040 139 133 133
q3 10643 1134 1153 1134
q4 10223 758 805 758
q5 7720 3177 3107 3107
q6 201 120 123 120
q7 855 492 481 481
q8 9227 1929 1926 1926
q9 7211 6375 6327 6327
q10 8220 3051 3089 3051
q11 420 222 213 213
q12 348 188 186 186
q13 17982 3328 3339 3328
q14 247 216 214 214
q15 538 507 507 507
q16 470 366 369 366
q17 944 519 486 486
q18 7334 6768 6897 6768
q19 3240 1325 1372 1325
q20 613 317 299 299
q21 2764 2336 2379 2336
q22 348 307 310 307
Total cold run time: 109209 ms
Total hot run time: 38709 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5523 5315 5263 5263
q2 319 221 209 209
q3 3335 3253 3246 3246
q4 2070 2088 2057 2057
q5 5990 5896 5943 5896
q6 195 119 117 117
q7 2306 1886 1904 1886
q8 3204 3354 3385 3354
q9 8938 8910 8806 8806
q10 3915 3839 3848 3839
q11 563 458 445 445
q12 814 613 621 613
q13 16942 3143 3137 3137
q14 283 255 287 255
q15 540 508 502 502
q16 507 446 466 446
q17 1865 1857 1892 1857
q18 9571 18455 9513 9513
q19 25351 1628 1517 1517
q20 4595 1931 1939 1931
q21 14380 5408 5375 5375
q22 997 503 572 503
Total cold run time: 112203 ms
Total hot run time: 60767 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1914064534
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](common) If the properties in DDL is not provided, the mysql client will lost connection. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30256:
URL: https://github.com/apache/doris/pull/30256#issuecomment-1914064579
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org