You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "linrrzqqq (via GitHub)" <gi...@apache.org> on 2026/10/09 06:23:08 UTC
[PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
linrrzqqq opened a new pull request, #68816:
URL: https://github.com/apache/doris/pull/68816
### What problem does this PR solve?
Issue Number: None
Problem Summary:
AI scalar functions, EMBED and AI\_AGG recreate HTTP clients for each request sequence, losing connection reuse between batches and Blocks.
This PR introduces `AIRequestExecutor` with exclusive client borrowing and a bounded idle cache. Scalar functions retain the cache in their execution contexts, while AI\_AGG shares it across local groups, flushes and resets.
It also clears responses between attempts, detaches cancellation callbacks before returning clients, sets JSON headers for LOCAL providers without an API key, and fixes `max_retries` to count retries after the initial request.
Unity compilation issues are fixed by adding missing protobuf and RuntimeState includes and locally suppressing CLucene conversion warnings, without changing CMakeLists.
### Release note
AI functions reuse HTTP connections across batches and Blocks. `max_retries=0` performs one request; `max_retries=3` allows up to four attempts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6077571907
<details>
<summary>TPC-DS: <b>Total hot run time: 152018 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0bcaf2a5664ada36f72511714b231da160f48d29, data reload: false
query5 4304 599 451 451
query6 417 203 192 192
query7 4806 441 223 223
query8 307 180 157 157
query9 8790 4035 3950 3950
query10 460 318 259 259
query11 5977 3405 3180 3180
query12 139 87 81 81
query13 1241 436 301 301
query14 6558 4776 4455 4455
query14_1 4258 4259 4231 4231
query15 209 208 188 188
query16 942 428 386 386
query17 860 669 553 553
query18 2414 428 312 312
query19 198 165 133 133
query20 80 81 78 78
query21 214 134 111 111
query22 12981 12889 12847 12847
query23 13191 12722 12112 12112
query23_1 12288 12281 12131 12131
query24 7108 823 474 474
query24_1 476 472 463 463
query25 531 424 361 361
query26 1311 253 139 139
query27 2761 452 268 268
query28 4598 1946 1956 1946
query29 1571 592 460 460
query30 299 222 181 181
query31 893 773 659 659
query32 147 95 94 94
query33 514 301 242 242
query34 929 845 492 492
query35 756 747 650 650
query36 821 815 740 740
query37 127 99 86 86
query38 1788 1753 1684 1684
query39 706 711 675 675
query39_1 680 679 647 647
query40 215 122 98 98
query41 71 69 67 67
query42 94 83 85 83
query43 361 382 326 326
query44 1296 699 703 699
query45 191 173 167 167
query46 832 963 565 565
query47 2975 3015 2953 2953
query48 308 314 271 271
query49 563 392 284 284
query50 691 283 209 209
query51 10555 10461 10590 10461
query52 78 79 68 68
query53 180 210 163 163
query54 243 188 178 178
query55 74 75 63 63
query56 215 206 213 206
query57 1607 1576 1543 1543
query58 278 256 249 249
query59 2354 2375 2121 2121
query60 287 232 223 223
query61 145 151 139 139
query62 396 339 284 284
query63 206 170 167 167
query64 2775 960 757 757
query65 3435 3351 3386 3351
query66 1779 422 303 303
query67 19992 20156 20072 20072
query68 3182 981 604 604
query69 384 280 268 268
query70 928 914 811 811
query71 297 218 211 211
query72 2889 2566 2300 2300
query73 534 538 300 300
query74 4575 4497 4273 4273
query75 2335 2269 1933 1933
query76 2406 1001 614 614
query77 365 395 296 296
query78 9004 9076 8556 8556
query79 984 850 505 505
query80 781 408 354 354
query81 585 325 273 273
query82 581 136 115 115
query83 283 193 182 182
query84 309 133 105 105
query85 979 431 373 373
query86 347 240 233 233
query87 2011 1968 1825 1825
query88 3599 2680 2695 2680
query89 346 291 257 257
query90 1713 188 186 186
query91 160 150 122 122
query92 100 80 90 80
query93 1013 975 557 557
query94 505 314 275 275
query95 573 345 321 321
query96 663 520 234 234
query97 2445 2420 2323 2323
query98 157 154 145 145
query99 772 767 644 644
Total cold run time: 234379 ms
Total hot run time: 152018 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #68816:
URL: https://github.com/apache/doris/pull/68816#discussion_r4227349904
##########
be/src/exprs/function/ai/ai_request_executor.cpp:
##########
@@ -0,0 +1,144 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exprs/function/ai/ai_request_executor.h"
+
+#include <unistd.h>
+
+#include <mutex>
+#include <vector>
+
+#include "exprs/function/ai/ai_adapter.h"
+#include "runtime/query_context.h"
+#include "service/http/http_client.h"
+#include "util/defer_op.h"
+#include "util/security.h"
+
+namespace doris {
+namespace {
+
+// Execute one attempt using the caller's borrowed client.
+Status send_request(HttpClient* client, const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter, QueryContext* query_context,
+ bool fail_on_http_error) {
+ // Discard any error body or partial response from the preceding attempt.
+ response.clear();
+ // init() resets request options/headers while preserving the connection cache.
+ RETURN_IF_ERROR(client->init(config.endpoint, fail_on_http_error));
+ if (query_context == nullptr) {
+ return Status::InternalError("Query context is null");
+ }
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+
+ const int64_t remaining_query_time = query_context->get_remaining_query_time_seconds();
+ if (remaining_query_time <= 0) {
+ return Status::TimedOut("Query timeout exceeded before AI request");
+ }
+ client->set_timeout_ms(remaining_query_time * 1000);
+
+ // Adapters also set provider headers, including JSON Content-Type for LOCAL.
+ RETURN_IF_ERROR(adapter.set_authentication(client));
+
+ // Rebind after init(), then detach before returning the client to the cache.
+ client->set_abort_callback([query_context] { return query_context->is_cancelled(); });
+
+ Status status = client->execute_post_request(request_body, &response);
+ client->set_abort_callback({});
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (!status.ok()) {
+ LOG(INFO) << "AI HTTP request failed before status validation, provider="
+ << config.provider_type << ", model=" << config.model_name
+ << ", endpoint=" << mask_token(config.endpoint)
+ << ", exec_status=" << status.to_string() << ", response_body=" << response;
+ return status;
+ }
+
+ const long http_status = client->get_http_status();
+ // Preserve scalar/EMBED non-200 errors; AI_AGG uses curl's error mode.
+ if (!fail_on_http_error && http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}, response_body={}",
+ http_status, mask_token(config.endpoint), response);
+ }
+ return Status::OK();
+}
+
+} // namespace
+
+// Borrowing removes a client from idle, ensuring exclusive use of its easy handle.
+struct AIRequestExecutor::ClientCache {
+ std::unique_ptr<HttpClient> borrow() {
+ {
+ std::lock_guard lock(mutex);
+ if (!idle.empty()) {
+ auto client = std::move(idle.back());
+ idle.pop_back();
+ return client;
+ }
+ }
+ return std::make_unique<HttpClient>();
+ }
+
+ void release(std::unique_ptr<HttpClient> client) {
+ // Bound idle clients, not TCP connections or in-flight requests.
+ // Excess clients are destroyed after releasing the lock.
+ constexpr size_t max_idle_clients = 8;
+ std::lock_guard lock(mutex);
+ if (idle.size() < max_idle_clients) {
+ idle.push_back(std::move(client));
+ }
+ }
+
+ std::mutex mutex;
+ std::vector<std::unique_ptr<HttpClient>> idle;
+};
+
+AIRequestExecutor::AIRequestExecutor() : _clients(std::make_unique<ClientCache>()) {}
+AIRequestExecutor::~AIRequestExecutor() = default;
+
+Status AIRequestExecutor::execute(const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter,
+ QueryContext* query_context, bool fail_on_http_error) {
+ auto client = _clients->borrow();
+ // Return the client on every exit path; network I/O holds no cache lock.
+ Defer return_client([&] { _clients->release(std::move(client)); });
+ Status status;
+ for (int64_t attempt = 0; attempt <= config.max_retries; ++attempt) {
+ status = send_request(client.get(), request_body, response, config, adapter, query_context,
+ fail_on_http_error);
+ if (query_context != nullptr && query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (status.ok()) {
+ const long http_status = client->get_http_status();
+ // Preserve AI_AGG's immediate rejection of transport-successful non-200.
+ if (http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}",
+ http_status, mask_token(config.endpoint));
+ }
+ return status;
+ }
+ // Retain the old fixed-delay policy, including sleep after the final failure.
+ sleep(config.retry_delay_second);
Review Comment:
[P2] Skip the backoff when no retry remains. With `max_retries=0`, this loop now correctly makes one request, but a fast failure still sleeps for `retry_delay_second` before returning. For example, a 503 with a 30-second delay holds the query for 30 seconds despite having no retries; cancellation during that sleep cannot wake it, and the caller receives the earlier 503 afterward. Sleep only before another attempt, and let that wait observe cancellation or the query deadline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6142350466
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/264)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/68816_9ba12bb86d33c47dfe9ce6df813f1b8b1a0812da_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/68816_9ba12bb86d33c47dfe9ce6df813f1b8b1a0812da_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6129274223
# BE Regression && UT Coverage Report
Increment line coverage `95.83% (23/24)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/68816_cb62d064458994428a18dcbaef7c256f99e13ddb_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/68816_cb62d064458994428a18dcbaef7c256f99e13ddb_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.92% (35472/46114) |
| Line Coverage | 62.18% (401336/645392) |
| Region Coverage | 58.64% (338990/578088) |
| Branch Coverage | 59.51% (156019/262157) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6125542271
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6078366347
# BE UT Coverage Report
Increment line coverage `100.00% (28/28)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/0bcaf2a5664ada36f72511714b231da160f48d29_0bcaf2a5664ada36f72511714b231da160f48d29/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/0bcaf2a5664ada36f72511714b231da160f48d29_0bcaf2a5664ada36f72511714b231da160f48d29/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 65.23% (30940/47434) |
| Line Coverage | 50.01% (325261/650373) |
| Region Coverage | 45.46% (262969/578502) |
| Branch Coverage | 47.14% (123443/261846) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6076316364
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6135828159
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6140137172
<details>
<summary>ClickBench: <b>Total hot run time: 24.57 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9ba12bb86d33c47dfe9ce6df813f1b8b1a0812da, data reload: false
query1 0.00 0.00 0.01
query2 0.10 0.05 0.05
query3 0.26 0.12 0.12
query4 1.61 0.13 0.13
query5 0.28 0.25 0.25
query6 1.17 0.69 0.68
query7 0.04 0.00 0.01
query8 0.05 0.04 0.03
query9 0.42 0.37 0.36
query10 0.62 0.62 0.59
query11 0.22 0.14 0.15
query12 0.19 0.15 0.15
query13 0.50 0.49 0.52
query14 0.98 0.97 0.97
query15 0.63 0.60 0.61
query16 0.36 0.35 0.31
query17 1.16 1.13 1.18
query18 0.23 0.21 0.22
query19 2.11 1.94 2.03
query20 0.02 0.02 0.01
query21 15.49 0.27 0.15
query22 4.41 0.06 0.06
query23 16.26 0.32 0.12
query24 3.13 0.91 0.40
query25 0.11 0.07 0.06
query26 1.44 0.25 0.14
query27 0.05 0.04 0.05
query28 2.16 0.63 0.40
query29 12.52 4.19 3.34
query30 0.28 0.14 0.13
query31 2.77 0.58 0.34
query32 3.22 0.62 0.51
query33 3.26 3.15 3.38
query34 15.46 4.04 3.42
query35 3.38 3.36 3.39
query36 0.56 0.43 0.48
query37 0.09 0.06 0.07
query38 0.05 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.14 0.14
query41 0.08 0.03 0.03
query42 0.03 0.04 0.03
query43 0.04 0.03 0.03
Total cold run time: 95.95 s
Total hot run time: 24.57 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6137769450
<details>
<summary>TPC-DS: <b>Total hot run time: 152384 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fee31fbb80a0834db6eecf3b91864deed9fb7fb0, data reload: false
query5 4303 609 461 461
query6 421 204 194 194
query7 4804 438 217 217
query8 316 181 167 167
query9 8775 3955 3984 3955
query10 478 315 258 258
query11 6039 3402 3153 3153
query12 140 88 82 82
query13 1242 445 345 345
query14 6532 4708 4476 4476
query14_1 4202 4275 4246 4246
query15 205 205 179 179
query16 949 449 411 411
query17 852 659 534 534
query18 2422 421 318 318
query19 195 159 135 135
query20 81 79 78 78
query21 210 135 111 111
query22 13025 12956 12791 12791
query23 13202 12675 12300 12300
query23_1 12336 12261 12320 12261
query24 7109 829 475 475
query24_1 481 477 472 472
query25 534 421 362 362
query26 1300 256 140 140
query27 2775 449 275 275
query28 4635 1966 1971 1966
query29 1539 595 461 461
query30 301 222 183 183
query31 891 771 662 662
query32 142 91 96 91
query33 517 309 256 256
query34 1004 853 495 495
query35 764 751 669 669
query36 839 828 751 751
query37 129 105 93 93
query38 1790 1751 1713 1713
query39 729 747 677 677
query39_1 662 666 706 666
query40 222 119 105 105
query41 82 70 70 70
query42 88 90 85 85
query43 368 384 326 326
query44 1293 713 709 709
query45 186 175 162 162
query46 865 958 564 564
query47 3014 3002 2858 2858
query48 305 322 257 257
query49 562 387 289 289
query50 666 272 218 218
query51 10540 10598 10344 10344
query52 78 82 68 68
query53 188 203 150 150
query54 251 197 185 185
query55 74 71 63 63
query56 227 220 219 219
query57 1626 1593 1548 1548
query58 285 256 252 252
query59 2328 2377 2142 2142
query60 276 237 217 217
query61 143 147 147 147
query62 385 338 290 290
query63 195 159 166 159
query64 2720 943 751 751
query65 3411 3335 3372 3335
query66 1767 408 295 295
query67 20273 20165 19983 19983
query68 3042 1013 630 630
query69 390 290 256 256
query70 942 836 889 836
query71 346 236 210 210
query72 2726 2596 2370 2370
query73 529 531 291 291
query74 4552 4477 4298 4298
query75 2335 2305 1931 1931
query76 2301 1016 621 621
query77 356 412 303 303
query78 9171 9048 8588 8588
query79 937 899 519 519
query80 540 423 337 337
query81 533 320 274 274
query82 572 133 111 111
query83 269 201 180 180
query84 309 120 102 102
query85 787 418 371 371
query86 313 238 234 234
query87 2009 1954 1843 1843
query88 3603 2692 2649 2649
query89 334 296 258 258
query90 1906 181 174 174
query91 165 150 125 125
query92 88 84 88 84
query93 999 999 574 574
query94 438 311 270 270
query95 583 334 397 334
query96 665 520 238 238
query97 2465 2444 2358 2358
query98 156 151 146 146
query99 766 761 660 660
Total cold run time: 234243 ms
Total hot run time: 152384 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6128119534
# BE UT Coverage Report
Increment line coverage `100.00% (28/28)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/cb62d064458994428a18dcbaef7c256f99e13ddb_cb62d064458994428a18dcbaef7c256f99e13ddb/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/cb62d064458994428a18dcbaef7c256f99e13ddb_cb62d064458994428a18dcbaef7c256f99e13ddb/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 65.69% (31293/47637) |
| Line Coverage | 50.47% (329421/652746) |
| Region Coverage | 45.85% (266152/580512) |
| Branch Coverage | 47.53% (125007/262980) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6126921243
<details>
<summary>TPC-H: <b>Total hot run time: 29329 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb62d064458994428a18dcbaef7c256f99e13ddb, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17598 5663 5562 5562
q2 2073 310 258 258
q3 10272 1427 872 872
q4 4675 483 359 359
q5 7447 846 539 539
q6 195 193 165 165
q7 773 798 612 612
q8 9253 1330 1129 1129
q9 5736 4482 4455 4455
q10 6831 1294 1044 1044
q11 445 261 238 238
q12 625 422 295 295
q13 18020 3088 2362 2362
q14 270 260 247 247
q15 q16 729 732 672 672
q17 1172 743 585 585
q18 7082 6237 6195 6195
q19 1102 959 630 630
q20 382 353 242 242
q21 5221 2578 2589 2578
q22 394 343 290 290
Total cold run time: 100295 ms
Total hot run time: 29329 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6640 6508 6634 6508
q2 722 618 586 586
q3 5496 5752 5207 5207
q4 2068 2482 1608 1608
q5 5820 5658 5545 5545
q6 296 210 160 160
q7 2241 2097 1844 1844
q8 3151 2819 2718 2718
q9 8253 7924 7714 7714
q10 3666 3632 3249 3249
q11 610 427 412 412
q12 692 713 501 501
q13 2774 3115 2374 2374
q14 280 294 264 264
q15 q16 670 688 609 609
q17 7805 7094 6921 6921
q18 13110 12246 13094 12246
q19 953 812 819 812
q20 2196 2169 1964 1964
q21 5917 4847 4928 4847
q22 497 484 399 399
Total cold run time: 73857 ms
Total hot run time: 66488 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6139923415
<details>
<summary>TPC-H: <b>Total hot run time: 29376 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ba12bb86d33c47dfe9ce6df813f1b8b1a0812da, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17619 5672 5587 5587
q2 2124 328 274 274
q3 10192 1473 845 845
q4 4692 473 358 358
q5 7451 799 545 545
q6 189 185 155 155
q7 786 804 616 616
q8 9246 1330 1102 1102
q9 5691 4474 4456 4456
q10 6832 1301 1044 1044
q11 440 253 238 238
q12 633 411 293 293
q13 18046 3115 2404 2404
q14 269 267 244 244
q15 q16 740 723 669 669
q17 1329 749 591 591
q18 7156 6282 6244 6244
q19 1109 957 591 591
q20 376 338 225 225
q21 5939 2585 2616 2585
q22 383 356 310 310
Total cold run time: 101242 ms
Total hot run time: 29376 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6688 6712 6796 6712
q2 793 600 572 572
q3 5576 5780 5259 5259
q4 2059 2162 1593 1593
q5 5816 5710 5981 5710
q6 248 189 146 146
q7 2275 2030 1809 1809
q8 3218 2743 2750 2743
q9 8252 8239 8142 8142
q10 3674 3582 3250 3250
q11 598 402 374 374
q12 660 714 514 514
q13 2764 3086 2381 2381
q14 279 287 274 274
q15 q16 678 677 613 613
q17 7862 7034 6818 6818
q18 13124 12266 13104 12266
q19 925 822 789 789
q20 2192 2173 1938 1938
q21 5841 4739 4794 4739
q22 523 445 398 398
Total cold run time: 74045 ms
Total hot run time: 67040 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6136915377
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6075644655
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6127104073
<details>
<summary>TPC-DS: <b>Total hot run time: 152856 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb62d064458994428a18dcbaef7c256f99e13ddb, data reload: false
query5 4312 571 435 435
query6 439 213 189 189
query7 4812 443 224 224
query8 321 178 166 166
query9 8758 4081 4044 4044
query10 491 323 263 263
query11 5992 3428 3187 3187
query12 152 91 88 88
query13 1282 462 325 325
query14 6544 4818 4513 4513
query14_1 4256 4293 4335 4293
query15 210 199 183 183
query16 933 452 402 402
query17 875 662 551 551
query18 2426 425 323 323
query19 191 175 134 134
query20 81 81 79 79
query21 212 150 118 118
query22 13015 13077 12987 12987
query23 13329 12765 12290 12290
query23_1 12467 12252 12245 12245
query24 7102 869 473 473
query24_1 485 518 487 487
query25 539 427 372 372
query26 1314 258 143 143
query27 2765 445 278 278
query28 4591 1963 1952 1952
query29 1591 611 456 456
query30 298 231 186 186
query31 911 782 652 652
query32 138 93 101 93
query33 514 307 276 276
query34 867 855 501 501
query35 767 781 657 657
query36 814 791 754 754
query37 140 103 94 94
query38 1810 1784 1714 1714
query39 724 721 692 692
query39_1 658 668 665 665
query40 223 123 101 101
query41 73 71 70 70
query42 88 86 88 86
query43 366 398 325 325
query44 1327 711 728 711
query45 179 185 160 160
query46 848 989 596 596
query47 2991 3044 2938 2938
query48 336 306 218 218
query49 563 391 300 300
query50 686 273 219 219
query51 10664 10385 10582 10385
query52 81 89 71 71
query53 186 204 150 150
query54 234 190 191 190
query55 77 71 64 64
query56 228 211 213 211
query57 1626 1617 1559 1559
query58 285 257 242 242
query59 2332 2387 2107 2107
query60 304 240 228 228
query61 148 145 145 145
query62 402 346 295 295
query63 202 164 169 164
query64 2747 934 800 800
query65 3436 3379 3388 3379
query66 1766 418 308 308
query67 20184 20226 19835 19835
query68 3117 1015 615 615
query69 423 302 252 252
query70 955 884 849 849
query71 309 245 214 214
query72 2686 2616 2336 2336
query73 520 529 305 305
query74 4596 4504 4319 4319
query75 2336 2314 1967 1967
query76 2311 1031 631 631
query77 365 393 298 298
query78 8968 9025 8502 8502
query79 938 911 544 544
query80 754 445 354 354
query81 568 321 271 271
query82 587 143 107 107
query83 283 196 174 174
query84 306 118 107 107
query85 980 418 356 356
query86 350 237 232 232
query87 2010 1952 1832 1832
query88 3605 2693 2704 2693
query89 348 300 254 254
query90 1704 184 179 179
query91 162 143 124 124
query92 94 88 86 86
query93 1000 980 572 572
query94 502 305 292 292
query95 582 336 396 336
query96 679 528 237 237
query97 2486 2434 2336 2336
query98 159 152 150 150
query99 774 752 659 659
Total cold run time: 234871 ms
Total hot run time: 152856 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6138890672
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6137642829
<details>
<summary>TPC-H: <b>Total hot run time: 29162 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fee31fbb80a0834db6eecf3b91864deed9fb7fb0, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17619 5635 5572 5572
q2 2107 311 253 253
q3 10233 1422 860 860
q4 4676 472 355 355
q5 7430 802 532 532
q6 188 187 157 157
q7 750 806 611 611
q8 9236 1294 1108 1108
q9 5931 4554 4447 4447
q10 6849 1298 1013 1013
q11 445 262 240 240
q12 635 397 283 283
q13 18061 3067 2373 2373
q14 269 272 248 248
q15 q16 760 730 684 684
q17 1172 740 580 580
q18 7155 6236 6206 6206
q19 1099 955 601 601
q20 405 335 222 222
q21 5404 2540 2589 2540
q22 417 358 277 277
Total cold run time: 100841 ms
Total hot run time: 29162 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6543 6496 6468 6468
q2 710 596 560 560
q3 5613 5759 5179 5179
q4 2345 2183 1630 1630
q5 5812 5779 5607 5607
q6 262 200 147 147
q7 2358 2068 1757 1757
q8 3098 2812 2702 2702
q9 8416 7936 7773 7773
q10 3677 3609 3214 3214
q11 624 425 371 371
q12 669 702 502 502
q13 2775 3066 2401 2401
q14 288 286 267 267
q15 q16 677 675 605 605
q17 7794 6928 6866 6866
q18 13225 12259 13060 12259
q19 931 786 801 786
q20 2192 2182 1955 1955
q21 5793 4725 5050 4725
q22 495 451 401 401
Total cold run time: 74297 ms
Total hot run time: 66175 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6137123876
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6077643285
<details>
<summary>ClickBench: <b>Total hot run time: 24.66 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0bcaf2a5664ada36f72511714b231da160f48d29, data reload: false
query1 0.00 0.00 0.01
query2 0.09 0.05 0.05
query3 0.26 0.11 0.12
query4 1.61 0.12 0.13
query5 0.28 0.26 0.25
query6 1.16 0.69 0.68
query7 0.05 0.01 0.00
query8 0.06 0.04 0.04
query9 0.42 0.35 0.35
query10 0.58 0.57 0.57
query11 0.21 0.15 0.14
query12 0.18 0.14 0.14
query13 0.50 0.49 0.49
query14 0.99 0.97 0.97
query15 0.64 0.60 0.62
query16 0.34 0.32 0.31
query17 1.17 1.15 1.12
query18 0.23 0.21 0.21
query19 2.14 2.00 2.04
query20 0.02 0.01 0.01
query21 15.48 0.29 0.14
query22 4.52 0.06 0.06
query23 16.26 0.31 0.13
query24 2.75 0.60 0.53
query25 0.12 0.07 0.05
query26 1.64 0.26 0.15
query27 0.06 0.04 0.04
query28 2.43 0.60 0.40
query29 12.48 4.08 3.24
query30 0.28 0.14 0.13
query31 2.78 0.58 0.35
query32 3.22 0.61 0.52
query33 3.07 3.31 3.21
query34 15.49 3.97 3.40
query35 3.33 3.36 3.34
query36 0.55 0.46 0.46
query37 0.11 0.07 0.06
query38 0.04 0.04 0.03
query39 0.04 0.03 0.03
query40 0.17 0.15 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 95.91 s
Total hot run time: 24.66 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6140063082
<details>
<summary>TPC-DS: <b>Total hot run time: 152842 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ba12bb86d33c47dfe9ce6df813f1b8b1a0812da, data reload: false
query5 4308 596 490 490
query6 452 211 189 189
query7 4807 445 221 221
query8 317 180 163 163
query9 8784 4021 4000 4000
query10 443 318 261 261
query11 5944 3433 3215 3215
query12 141 92 88 88
query13 1291 438 324 324
query14 6559 4801 4541 4541
query14_1 4235 4256 4240 4240
query15 209 199 194 194
query16 954 435 435 435
query17 867 660 549 549
query18 2411 427 341 341
query19 208 165 153 153
query20 83 79 82 79
query21 223 136 116 116
query22 13124 13002 12863 12863
query23 13258 12614 12139 12139
query23_1 12363 12217 12363 12217
query24 7080 829 479 479
query24_1 475 476 482 476
query25 544 426 370 370
query26 1249 255 138 138
query27 2764 439 275 275
query28 4607 1971 1951 1951
query29 1569 610 471 471
query30 297 216 189 189
query31 880 763 672 672
query32 164 92 102 92
query33 508 313 245 245
query34 914 857 518 518
query35 758 800 654 654
query36 815 809 749 749
query37 128 106 90 90
query38 1810 1764 1721 1721
query39 708 713 670 670
query39_1 681 666 668 666
query40 224 121 101 101
query41 71 69 71 69
query42 91 84 84 84
query43 368 390 328 328
query44 1332 713 702 702
query45 176 174 173 173
query46 840 955 577 577
query47 3044 2890 2971 2890
query48 291 325 229 229
query49 563 386 292 292
query50 694 277 212 212
query51 10732 10812 10517 10517
query52 81 80 71 71
query53 193 207 155 155
query54 235 185 177 177
query55 74 72 64 64
query56 241 212 214 212
query57 1634 1583 1579 1579
query58 286 258 245 245
query59 2363 2400 2140 2140
query60 282 238 217 217
query61 147 163 136 136
query62 395 340 293 293
query63 202 161 161 161
query64 2787 930 796 796
query65 3440 3367 3358 3358
query66 1785 412 305 305
query67 20138 20202 19956 19956
query68 2964 1037 618 618
query69 387 291 238 238
query70 914 872 841 841
query71 290 236 214 214
query72 2666 2571 2248 2248
query73 517 525 299 299
query74 4570 4471 4356 4356
query75 2345 2296 1952 1952
query76 2302 1018 603 603
query77 355 387 299 299
query78 9074 8991 8561 8561
query79 985 853 519 519
query80 733 436 340 340
query81 580 335 276 276
query82 898 136 124 124
query83 313 210 185 185
query84 321 122 104 104
query85 856 424 365 365
query86 362 238 232 232
query87 1999 1961 1821 1821
query88 3651 2733 2698 2698
query89 344 296 261 261
query90 1793 186 191 186
query91 160 156 128 128
query92 91 99 91 91
query93 998 991 576 576
query94 519 324 281 281
query95 605 432 331 331
query96 672 525 238 238
query97 2466 2435 2372 2372
query98 157 150 151 150
query99 777 772 663 663
Total cold run time: 235033 ms
Total hot run time: 152842 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6137827185
<details>
<summary>ClickBench: <b>Total hot run time: 24.33 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fee31fbb80a0834db6eecf3b91864deed9fb7fb0, data reload: false
query1 0.00 0.00 0.01
query2 0.10 0.05 0.05
query3 0.26 0.12 0.11
query4 1.63 0.14 0.12
query5 0.23 0.26 0.22
query6 1.17 0.69 0.67
query7 0.04 0.01 0.00
query8 0.05 0.03 0.04
query9 0.42 0.35 0.35
query10 0.60 0.56 0.59
query11 0.20 0.15 0.14
query12 0.18 0.14 0.15
query13 0.50 0.48 0.49
query14 1.00 0.96 0.95
query15 0.63 0.61 0.62
query16 0.34 0.35 0.31
query17 1.18 1.13 1.12
query18 0.22 0.22 0.21
query19 2.10 2.02 1.94
query20 0.02 0.02 0.02
query21 15.42 0.26 0.15
query22 4.47 0.06 0.06
query23 16.30 0.32 0.12
query24 3.19 0.59 0.37
query25 0.12 0.07 0.06
query26 1.09 0.24 0.15
query27 0.04 0.04 0.04
query28 1.74 0.61 0.39
query29 12.49 4.19 3.33
query30 0.29 0.14 0.12
query31 2.78 0.57 0.33
query32 3.22 0.62 0.51
query33 3.20 3.20 3.21
query34 15.48 4.04 3.39
query35 3.35 3.38 3.34
query36 0.55 0.44 0.40
query37 0.10 0.07 0.06
query38 0.04 0.04 0.03
query39 0.04 0.03 0.03
query40 0.16 0.14 0.14
query41 0.08 0.02 0.02
query42 0.04 0.03 0.03
query43 0.05 0.03 0.03
Total cold run time: 95.11 s
Total hot run time: 24.33 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs merged PR #68816:
URL: https://github.com/apache/doris/pull/68816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6166691988
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6166692598
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6076880560
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6079493155
# BE Regression && UT Coverage Report
Increment line coverage `91.67% (22/24)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/68816_0bcaf2a5664ada36f72511714b231da160f48d29_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/68816_0bcaf2a5664ada36f72511714b231da160f48d29_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.78% (35252/45912) |
| Line Coverage | 61.98% (398558/643038) |
| Region Coverage | 58.47% (336820/576099) |
| Branch Coverage | 59.30% (154780/261033) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6075579493
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6139332172
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6077432265
<details>
<summary>TPC-H: <b>Total hot run time: 29267 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0bcaf2a5664ada36f72511714b231da160f48d29, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17591 5640 5563 5563
q2 2134 322 271 271
q3 10189 1411 837 837
q4 4681 478 356 356
q5 7433 799 546 546
q6 196 192 159 159
q7 755 803 628 628
q8 9256 1328 1070 1070
q9 5741 4503 4463 4463
q10 6837 1310 1030 1030
q11 434 252 246 246
q12 628 403 286 286
q13 18067 3062 2399 2399
q14 278 270 249 249
q15 q16 740 726 678 678
q17 1213 744 587 587
q18 7135 6228 6193 6193
q19 1125 959 606 606
q20 397 344 226 226
q21 5798 2584 2685 2584
q22 411 383 290 290
Total cold run time: 101039 ms
Total hot run time: 29267 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6684 6690 6766 6690
q2 749 598 553 553
q3 5507 5736 5292 5292
q4 2046 2182 1603 1603
q5 5807 5758 5826 5758
q6 243 189 147 147
q7 2272 2058 1851 1851
q8 3151 2805 2750 2750
q9 8185 8144 7969 7969
q10 3654 3577 3271 3271
q11 605 405 377 377
q12 668 706 505 505
q13 2739 3078 2410 2410
q14 289 283 263 263
q15 q16 668 688 612 612
q17 7821 6999 6844 6844
q18 13141 12338 13054 12338
q19 921 796 816 796
q20 2181 2168 1932 1932
q21 5838 4715 4785 4715
q22 500 462 414 414
Total cold run time: 73669 ms
Total hot run time: 67090 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6129465773
# FE Regression Coverage Report
Increment line coverage ` 100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/68816_cb62d064458994428a18dcbaef7c256f99e13ddb_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/68816_cb62d064458994428a18dcbaef7c256f99e13ddb_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "linrrzqqq (via GitHub)" <gi...@apache.org>.
linrrzqqq commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6125544283
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68816:
URL: https://github.com/apache/doris/pull/68816#issuecomment-6127189642
<details>
<summary>ClickBench: <b>Total hot run time: 24.74 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb62d064458994428a18dcbaef7c256f99e13ddb, data reload: false
query1 0.01 0.01 0.01
query2 0.09 0.05 0.05
query3 0.27 0.12 0.11
query4 1.61 0.13 0.12
query5 0.27 0.25 0.24
query6 1.17 0.68 0.69
query7 0.04 0.01 0.01
query8 0.05 0.04 0.04
query9 0.42 0.35 0.35
query10 0.58 0.56 0.57
query11 0.22 0.14 0.14
query12 0.18 0.15 0.14
query13 0.51 0.49 0.48
query14 0.98 0.97 0.97
query15 0.62 0.61 0.61
query16 0.33 0.36 0.35
query17 1.16 1.16 1.14
query18 0.22 0.21 0.21
query19 2.12 2.01 1.99
query20 0.02 0.02 0.01
query21 15.49 0.25 0.14
query22 4.43 0.05 0.06
query23 16.26 0.32 0.12
query24 2.89 0.72 0.41
query25 0.11 0.08 0.06
query26 1.79 0.25 0.15
query27 0.04 0.04 0.04
query28 2.54 0.61 0.41
query29 12.52 4.23 3.32
query30 0.28 0.14 0.12
query31 2.77 0.58 0.34
query32 3.22 0.62 0.52
query33 3.23 3.28 3.30
query34 15.45 4.02 3.41
query35 3.37 3.35 3.35
query36 0.54 0.49 0.46
query37 0.09 0.07 0.06
query38 0.06 0.03 0.04
query39 0.04 0.03 0.03
query40 0.17 0.15 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.05 0.03 0.03
Total cold run time: 96.33 s
Total hot run time: 24.74 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #68816:
URL: https://github.com/apache/doris/pull/68816#discussion_r4249017936
##########
be/test/service/http/http_client_test.cpp:
##########
@@ -46,10 +56,290 @@
#include "service/http/utils.h"
#include "service/http_service.h"
#include "testutil/http_utils.h"
+#include "util/defer_op.h"
#include "util/md5.h"
namespace doris {
+class HttpClientTlsTest : public testing::Test {
+protected:
+ void SetUp() override {
+ {
+ std::scoped_lock lock(*config::get_mutable_string_config_lock());
+ _original_ca_paths = config::ca_cert_file_paths;
+ }
+ EVP_PKEY* raw_key = nullptr;
+ generate_key(&raw_key);
+ ASSERT_FALSE(HasFatalFailure());
+ std::unique_ptr<EVP_PKEY, decltype(&EVP_PKEY_free)> key(raw_key, EVP_PKEY_free);
+ std::unique_ptr<X509, decltype(&X509_free)> certificate(X509_new(), X509_free);
+ ASSERT_NE(certificate, nullptr);
+ generate_certificate(certificate.get(), key.get());
+ ASSERT_FALSE(HasFatalFailure());
+ start_server_and_write_ca(certificate.get(), key.get());
+ ASSERT_FALSE(HasFatalFailure());
+ }
+
+ static void generate_key(EVP_PKEY** key) {
+ std::unique_ptr<EVP_PKEY_CTX, decltype(&EVP_PKEY_CTX_free)> key_context(
+ EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, nullptr), EVP_PKEY_CTX_free);
+ ASSERT_NE(key_context, nullptr);
+ ASSERT_EQ(EVP_PKEY_keygen_init(key_context.get()), 1);
+ ASSERT_EQ(EVP_PKEY_CTX_set_rsa_keygen_bits(key_context.get(), 2048), 1);
+ ASSERT_EQ(EVP_PKEY_keygen(key_context.get(), key), 1);
+ }
+
+ static void generate_certificate(X509* certificate, EVP_PKEY* key) {
+ ASSERT_EQ(X509_set_version(certificate, 2), 1);
+ ASSERT_EQ(ASN1_INTEGER_set(X509_get_serialNumber(certificate), 1), 1);
+ ASSERT_NE(X509_gmtime_adj(X509_getm_notBefore(certificate), -60), nullptr);
+ ASSERT_NE(X509_gmtime_adj(X509_getm_notAfter(certificate), 3600), nullptr);
+ ASSERT_EQ(X509_set_pubkey(certificate, key), 1);
+ auto* subject = X509_get_subject_name(certificate);
+ ASSERT_EQ(X509_NAME_add_entry_by_txt(subject, "CN", MBSTRING_ASC,
+ reinterpret_cast<const unsigned char*>("127.0.0.1"),
+ -1, -1, 0),
+ 1);
+ ASSERT_EQ(X509_set_issuer_name(certificate, subject), 1);
+ add_extension(certificate, NID_basic_constraints, "critical,CA:TRUE");
+ add_extension(certificate, NID_subject_alt_name, "IP:127.0.0.1");
+ ASSERT_FALSE(HasFatalFailure());
+ ASSERT_GT(X509_sign(certificate, key, EVP_sha256()), 0);
+ }
+
+ void start_server_and_write_ca(X509* certificate, EVP_PKEY* key) {
+ _ssl_context.reset(SSL_CTX_new(TLS_server_method()));
+ ASSERT_NE(_ssl_context, nullptr);
+ ASSERT_EQ(SSL_CTX_use_certificate(_ssl_context.get(), certificate), 1);
+ ASSERT_EQ(SSL_CTX_use_PrivateKey(_ssl_context.get(), key), 1);
+ std::unique_ptr<BIO, decltype(&BIO_free)> cert_bio(BIO_new(BIO_s_mem()), BIO_free);
+ ASSERT_NE(cert_bio, nullptr);
+ ASSERT_EQ(PEM_write_bio_X509(cert_bio.get(), certificate), 1);
+ BUF_MEM* cert_pem = nullptr;
+ BIO_get_mem_ptr(cert_bio.get(), &cert_pem);
+ char ca_path[] = "/tmp/doris-http-client-ca-XXXXXX";
+ const int fd = mkstemp(ca_path);
+ ASSERT_GE(fd, 0);
+ _ca_path = ca_path;
+ Defer close_file {[fd] { close(fd); }};
+ ASSERT_EQ(write(fd, cert_pem->data, cert_pem->length),
+ static_cast<ssize_t>(cert_pem->length));
+ start_listener();
+ ASSERT_FALSE(HasFatalFailure());
+ }
+
+ void start_listener() {
+ _listen_fd = socket(AF_INET, SOCK_STREAM, 0);
+ ASSERT_GE(_listen_fd, 0);
+ sockaddr_in address {};
+ address.sin_family = AF_INET;
+ address.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+ ASSERT_EQ(bind(_listen_fd, reinterpret_cast<sockaddr*>(&address), sizeof(address)), 0);
+ ASSERT_EQ(listen(_listen_fd, 8), 0);
+ socklen_t address_length = sizeof(address);
+ ASSERT_EQ(getsockname(_listen_fd, reinterpret_cast<sockaddr*>(&address), &address_length),
+ 0);
+ _port = ntohs(address.sin_port);
+ _server_thread = std::thread([this] { serve_https(); });
+ }
+
+ void serve_https() {
+ sigset_t blocked_signals;
+ sigemptyset(&blocked_signals);
+ sigaddset(&blocked_signals, SIGPIPE);
+ ASSERT_EQ(pthread_sigmask(SIG_BLOCK, &blocked_signals, nullptr), 0);
+ Defer drain_sigpipe {[&blocked_signals] {
+ const timespec no_wait {};
+ while (sigtimedwait(&blocked_signals, nullptr, &no_wait) == SIGPIPE) {
+ }
+ }};
+ while (!_stopped) {
+ const int fd = accept(_listen_fd, nullptr, nullptr);
+ if (fd < 0) {
+ return;
+ }
+ {
+ std::scoped_lock lock(_connection_mutex);
+ _connection_fd = fd;
+ if (_stopped) {
+ shutdown(fd, SHUT_RDWR);
+ }
+ }
+ Defer close_connection {[this, fd] {
+ std::scoped_lock lock(_connection_mutex);
+ _connection_fd = -1;
+ close(fd);
+ }};
+ respond_https(fd);
+ }
+ }
+
+ void respond_https(int fd) {
+ const timeval timeout {.tv_sec = 5, .tv_usec = 0};
+ ASSERT_EQ(setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout)), 0);
+ ASSERT_EQ(setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &timeout, sizeof(timeout)), 0);
+ std::unique_ptr<SSL, decltype(&SSL_free)> connection(SSL_new(_ssl_context.get()), SSL_free);
+ ASSERT_NE(connection, nullptr);
+ ASSERT_EQ(SSL_set_fd(connection.get(), fd), 1);
+ if (SSL_accept(connection.get()) != 1) {
+ return;
+ }
+ char request[4096];
+ if (SSL_read(connection.get(), request, sizeof(request)) > 0) {
+ constexpr std::string_view response =
+ "HTTP/1.1 200 OK\r\nContent-Length: 2\r\nConnection: close\r\n\r\nOK";
+ SSL_write(connection.get(), response.data(), response.size());
+ }
+ SSL_shutdown(connection.get());
+ }
+
+ void TearDown() override {
+ set_ca_paths(_original_ca_paths);
+ if (!_ca_path.empty()) {
+ EXPECT_EQ(unlink(_ca_path.c_str()), 0);
+ }
+ _stopped = true;
+ {
+ std::scoped_lock lock(_connection_mutex);
+ if (_connection_fd >= 0) {
+ shutdown(_connection_fd, SHUT_RDWR);
+ }
+ }
+ if (_listen_fd >= 0) {
+ shutdown(_listen_fd, SHUT_RDWR);
Review Comment:
[P2] Wake the listener explicitly before joining the TLS server. On macOS, `shutdown()` on this unconnected listening socket returns `ENOTCONN` without waking the worker's blocking `accept()` ([Darwin socket implementation](https://raw.githubusercontent.com/apple-oss-distributions/xnu/main/bsd/kern/uipc_socket.c)). For example, `tls_ca_path_probe_is_cached_and_configuration_invalidates_it` starts the server but never connects to it; if the worker has reached `accept()`, setting `_stopped` cannot interrupt that syscall, `_server_thread.join()` waits indefinitely, and the later `close()` is never reached. This remains after fixing the separate `sigtimedwait` compile issue. Use a portable wakeup or a bounded/nonblocking accept loop before joining.
##########
be/test/service/http/http_client_test.cpp:
##########
@@ -46,10 +56,290 @@
#include "service/http/utils.h"
#include "service/http_service.h"
#include "testutil/http_utils.h"
+#include "util/defer_op.h"
#include "util/md5.h"
namespace doris {
+class HttpClientTlsTest : public testing::Test {
+protected:
+ void SetUp() override {
+ {
+ std::scoped_lock lock(*config::get_mutable_string_config_lock());
+ _original_ca_paths = config::ca_cert_file_paths;
+ }
+ EVP_PKEY* raw_key = nullptr;
+ generate_key(&raw_key);
+ ASSERT_FALSE(HasFatalFailure());
+ std::unique_ptr<EVP_PKEY, decltype(&EVP_PKEY_free)> key(raw_key, EVP_PKEY_free);
+ std::unique_ptr<X509, decltype(&X509_free)> certificate(X509_new(), X509_free);
+ ASSERT_NE(certificate, nullptr);
+ generate_certificate(certificate.get(), key.get());
+ ASSERT_FALSE(HasFatalFailure());
+ start_server_and_write_ca(certificate.get(), key.get());
+ ASSERT_FALSE(HasFatalFailure());
+ }
+
+ static void generate_key(EVP_PKEY** key) {
+ std::unique_ptr<EVP_PKEY_CTX, decltype(&EVP_PKEY_CTX_free)> key_context(
+ EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, nullptr), EVP_PKEY_CTX_free);
+ ASSERT_NE(key_context, nullptr);
+ ASSERT_EQ(EVP_PKEY_keygen_init(key_context.get()), 1);
+ ASSERT_EQ(EVP_PKEY_CTX_set_rsa_keygen_bits(key_context.get(), 2048), 1);
+ ASSERT_EQ(EVP_PKEY_keygen(key_context.get(), key), 1);
+ }
+
+ static void generate_certificate(X509* certificate, EVP_PKEY* key) {
+ ASSERT_EQ(X509_set_version(certificate, 2), 1);
+ ASSERT_EQ(ASN1_INTEGER_set(X509_get_serialNumber(certificate), 1), 1);
+ ASSERT_NE(X509_gmtime_adj(X509_getm_notBefore(certificate), -60), nullptr);
+ ASSERT_NE(X509_gmtime_adj(X509_getm_notAfter(certificate), 3600), nullptr);
+ ASSERT_EQ(X509_set_pubkey(certificate, key), 1);
+ auto* subject = X509_get_subject_name(certificate);
+ ASSERT_EQ(X509_NAME_add_entry_by_txt(subject, "CN", MBSTRING_ASC,
+ reinterpret_cast<const unsigned char*>("127.0.0.1"),
+ -1, -1, 0),
+ 1);
+ ASSERT_EQ(X509_set_issuer_name(certificate, subject), 1);
+ add_extension(certificate, NID_basic_constraints, "critical,CA:TRUE");
+ add_extension(certificate, NID_subject_alt_name, "IP:127.0.0.1");
+ ASSERT_FALSE(HasFatalFailure());
+ ASSERT_GT(X509_sign(certificate, key, EVP_sha256()), 0);
+ }
+
+ void start_server_and_write_ca(X509* certificate, EVP_PKEY* key) {
+ _ssl_context.reset(SSL_CTX_new(TLS_server_method()));
+ ASSERT_NE(_ssl_context, nullptr);
+ ASSERT_EQ(SSL_CTX_use_certificate(_ssl_context.get(), certificate), 1);
+ ASSERT_EQ(SSL_CTX_use_PrivateKey(_ssl_context.get(), key), 1);
+ std::unique_ptr<BIO, decltype(&BIO_free)> cert_bio(BIO_new(BIO_s_mem()), BIO_free);
+ ASSERT_NE(cert_bio, nullptr);
+ ASSERT_EQ(PEM_write_bio_X509(cert_bio.get(), certificate), 1);
+ BUF_MEM* cert_pem = nullptr;
+ BIO_get_mem_ptr(cert_bio.get(), &cert_pem);
+ char ca_path[] = "/tmp/doris-http-client-ca-XXXXXX";
+ const int fd = mkstemp(ca_path);
+ ASSERT_GE(fd, 0);
+ _ca_path = ca_path;
+ Defer close_file {[fd] { close(fd); }};
+ ASSERT_EQ(write(fd, cert_pem->data, cert_pem->length),
+ static_cast<ssize_t>(cert_pem->length));
+ start_listener();
+ ASSERT_FALSE(HasFatalFailure());
+ }
+
+ void start_listener() {
+ _listen_fd = socket(AF_INET, SOCK_STREAM, 0);
+ ASSERT_GE(_listen_fd, 0);
+ sockaddr_in address {};
+ address.sin_family = AF_INET;
+ address.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+ ASSERT_EQ(bind(_listen_fd, reinterpret_cast<sockaddr*>(&address), sizeof(address)), 0);
+ ASSERT_EQ(listen(_listen_fd, 8), 0);
+ socklen_t address_length = sizeof(address);
+ ASSERT_EQ(getsockname(_listen_fd, reinterpret_cast<sockaddr*>(&address), &address_length),
+ 0);
+ _port = ntohs(address.sin_port);
+ _server_thread = std::thread([this] { serve_https(); });
+ }
+
+ void serve_https() {
+ sigset_t blocked_signals;
+ sigemptyset(&blocked_signals);
+ sigaddset(&blocked_signals, SIGPIPE);
+ ASSERT_EQ(pthread_sigmask(SIG_BLOCK, &blocked_signals, nullptr), 0);
+ Defer drain_sigpipe {[&blocked_signals] {
+ const timespec no_wait {};
+ while (sigtimedwait(&blocked_signals, nullptr, &no_wait) == SIGPIPE) {
Review Comment:
[P2] Keep the TLS fixture's signal cleanup portable. On the macOS BE unit-test build, this unconditional `sigtimedwait` call fails to compile because Darwin does not provide it ([Apple signal declarations](https://raw.githubusercontent.com/apple-oss-distributions/Libc/main/include/signal.h)). `be/test/CMakeLists.txt` includes `service/*.cpp` on macOS, so this breaks the whole `doris_be_test` target, even when the TLS module is disabled. The repository already handles this exact platform limitation in `be/CMakeLists.txt` and `boost-1.81.0-mac-sigtimedwait.patch`. Guard this cleanup for supported platforms or provide a macOS-compatible SIGPIPE drain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #68816:
URL: https://github.com/apache/doris/pull/68816#discussion_r4236240171
##########
be/src/exprs/function/ai/ai_request_executor.cpp:
##########
@@ -0,0 +1,167 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exprs/function/ai/ai_request_executor.h"
+
+#include <algorithm>
+#include <chrono>
+#include <mutex>
+#include <thread>
+#include <vector>
+
+#include "exprs/function/ai/ai_adapter.h"
+#include "runtime/query_context.h"
+#include "service/http/http_client.h"
+#include "util/defer_op.h"
+#include "util/security.h"
+
+namespace doris {
+namespace {
+
+Status wait_before_retry(int32_t delay_seconds, QueryContext* query_context) {
+ const auto retry_at = std::chrono::steady_clock::now() + std::chrono::seconds(delay_seconds);
+ while (true) {
+ if (query_context != nullptr) {
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (query_context->get_remaining_query_time_seconds() <= 0) {
+ return Status::TimedOut("Query timeout exceeded during AI retry wait");
+ }
+ }
+ const auto now = std::chrono::steady_clock::now();
+ if (now >= retry_at) {
+ return Status::OK();
+ }
+ // Poll query state during backoff, including before the next attempt.
+ std::this_thread::sleep_until(std::min(retry_at, now + std::chrono::milliseconds(100)));
+ }
+}
+
+// Execute one attempt using the caller's borrowed client.
+Status send_request(HttpClient* client, const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter, QueryContext* query_context,
+ bool fail_on_http_error) {
+ // Discard any error body or partial response from the preceding attempt.
+ response.clear();
+ // init() resets request options/headers while preserving the connection cache.
+ RETURN_IF_ERROR(client->init(config.endpoint, fail_on_http_error));
+ if (query_context == nullptr) {
+ return Status::InternalError("Query context is null");
+ }
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+
+ const int64_t remaining_query_time = query_context->get_remaining_query_time_seconds();
+ if (remaining_query_time <= 0) {
+ return Status::TimedOut("Query timeout exceeded before AI request");
+ }
+ client->set_timeout_ms(remaining_query_time * 1000);
+
+ // Adapters also set provider headers, including JSON Content-Type for LOCAL.
+ RETURN_IF_ERROR(adapter.set_authentication(client));
+
+ // Rebind after init(), then detach before returning the client to the cache.
+ client->set_abort_callback([query_context] { return query_context->is_cancelled(); });
+
+ Status status = client->execute_post_request(request_body, &response);
+ client->set_abort_callback({});
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (!status.ok()) {
+ LOG(INFO) << "AI HTTP request failed before status validation, provider="
+ << config.provider_type << ", model=" << config.model_name
+ << ", endpoint=" << mask_token(config.endpoint)
+ << ", exec_status=" << status.to_string() << ", response_body=" << response;
+ return status;
+ }
+
+ const long http_status = client->get_http_status();
+ // Preserve scalar/EMBED non-200 errors; AI_AGG uses curl's error mode.
+ if (!fail_on_http_error && http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}, response_body={}",
+ http_status, mask_token(config.endpoint), response);
+ }
+ return Status::OK();
+}
+
+} // namespace
+
+// Borrowing removes a client from idle, ensuring exclusive use of its easy handle.
+struct AIRequestExecutor::ClientCache {
+ std::unique_ptr<HttpClient> borrow() {
+ {
+ std::lock_guard lock(mutex);
+ if (!idle.empty()) {
+ auto client = std::move(idle.back());
+ idle.pop_back();
+ return client;
+ }
+ }
+ return std::make_unique<HttpClient>();
+ }
+
+ void release(std::unique_ptr<HttpClient> client) {
+ // Bound idle clients, not TCP connections or in-flight requests.
+ // Excess clients are destroyed after releasing the lock.
+ constexpr size_t max_idle_clients = 8;
+ std::lock_guard lock(mutex);
+ if (idle.size() < max_idle_clients) {
+ idle.push_back(std::move(client));
+ }
+ }
+
+ std::mutex mutex;
+ std::vector<std::unique_ptr<HttpClient>> idle;
+};
+
+AIRequestExecutor::AIRequestExecutor() : _clients(std::make_unique<ClientCache>()) {}
+AIRequestExecutor::~AIRequestExecutor() = default;
+
+Status AIRequestExecutor::execute(const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter,
+ QueryContext* query_context, bool fail_on_http_error) {
+ auto client = _clients->borrow();
+ // Return the client on every exit path; network I/O holds no cache lock.
+ Defer return_client([&] { _clients->release(std::move(client)); });
+ Status status;
+ for (int64_t attempt = 0; attempt <= config.max_retries; ++attempt) {
+ status = send_request(client.get(), request_body, response, config, adapter, query_context,
+ fail_on_http_error);
+ if (query_context != nullptr && query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (status.ok()) {
+ const long http_status = client->get_http_status();
+ // Preserve AI_AGG's immediate rejection of transport-successful non-200.
+ if (http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}",
+ http_status, mask_token(config.endpoint));
+ }
+ return status;
+ }
+ if (attempt == config.max_retries) {
+ return status;
+ }
+ RETURN_IF_ERROR(wait_before_retry(config.retry_delay_second, query_context));
Review Comment:
Please change the default `ai.retry_delay_second` from `0` to `1` second (`AIProperties.DEFAULT_RETRY_DELAY_SECOND`), while preserving explicit user settings. With the current default and `max_retries=3`, a fast 429/503 can exhaust all four attempts back-to-back. A 1-second default gives transient failures some recovery time and avoids immediate retry bursts.
##########
be/src/exprs/function/ai/ai_request_executor.cpp:
##########
@@ -0,0 +1,167 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exprs/function/ai/ai_request_executor.h"
+
+#include <algorithm>
+#include <chrono>
+#include <mutex>
+#include <thread>
+#include <vector>
+
+#include "exprs/function/ai/ai_adapter.h"
+#include "runtime/query_context.h"
+#include "service/http/http_client.h"
+#include "util/defer_op.h"
+#include "util/security.h"
+
+namespace doris {
+namespace {
+
+Status wait_before_retry(int32_t delay_seconds, QueryContext* query_context) {
+ const auto retry_at = std::chrono::steady_clock::now() + std::chrono::seconds(delay_seconds);
+ while (true) {
+ if (query_context != nullptr) {
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (query_context->get_remaining_query_time_seconds() <= 0) {
+ return Status::TimedOut("Query timeout exceeded during AI retry wait");
+ }
+ }
+ const auto now = std::chrono::steady_clock::now();
+ if (now >= retry_at) {
+ return Status::OK();
+ }
+ // Poll query state during backoff, including before the next attempt.
+ std::this_thread::sleep_until(std::min(retry_at, now + std::chrono::milliseconds(100)));
+ }
+}
+
+// Execute one attempt using the caller's borrowed client.
+Status send_request(HttpClient* client, const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter, QueryContext* query_context,
+ bool fail_on_http_error) {
+ // Discard any error body or partial response from the preceding attempt.
+ response.clear();
+ // init() resets request options/headers while preserving the connection cache.
+ RETURN_IF_ERROR(client->init(config.endpoint, fail_on_http_error));
+ if (query_context == nullptr) {
+ return Status::InternalError("Query context is null");
+ }
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+
+ const int64_t remaining_query_time = query_context->get_remaining_query_time_seconds();
+ if (remaining_query_time <= 0) {
+ return Status::TimedOut("Query timeout exceeded before AI request");
+ }
+ client->set_timeout_ms(remaining_query_time * 1000);
+
+ // Adapters also set provider headers, including JSON Content-Type for LOCAL.
+ RETURN_IF_ERROR(adapter.set_authentication(client));
+
+ // Rebind after init(), then detach before returning the client to the cache.
+ client->set_abort_callback([query_context] { return query_context->is_cancelled(); });
+
+ Status status = client->execute_post_request(request_body, &response);
+ client->set_abort_callback({});
+ if (query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (!status.ok()) {
+ LOG(INFO) << "AI HTTP request failed before status validation, provider="
+ << config.provider_type << ", model=" << config.model_name
+ << ", endpoint=" << mask_token(config.endpoint)
+ << ", exec_status=" << status.to_string() << ", response_body=" << response;
+ return status;
+ }
+
+ const long http_status = client->get_http_status();
+ // Preserve scalar/EMBED non-200 errors; AI_AGG uses curl's error mode.
+ if (!fail_on_http_error && http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}, response_body={}",
+ http_status, mask_token(config.endpoint), response);
+ }
+ return Status::OK();
+}
+
+} // namespace
+
+// Borrowing removes a client from idle, ensuring exclusive use of its easy handle.
+struct AIRequestExecutor::ClientCache {
+ std::unique_ptr<HttpClient> borrow() {
+ {
+ std::lock_guard lock(mutex);
+ if (!idle.empty()) {
+ auto client = std::move(idle.back());
+ idle.pop_back();
+ return client;
+ }
+ }
+ return std::make_unique<HttpClient>();
+ }
+
+ void release(std::unique_ptr<HttpClient> client) {
+ // Bound idle clients, not TCP connections or in-flight requests.
+ // Excess clients are destroyed after releasing the lock.
+ constexpr size_t max_idle_clients = 8;
+ std::lock_guard lock(mutex);
+ if (idle.size() < max_idle_clients) {
+ idle.push_back(std::move(client));
+ }
+ }
+
+ std::mutex mutex;
+ std::vector<std::unique_ptr<HttpClient>> idle;
+};
+
+AIRequestExecutor::AIRequestExecutor() : _clients(std::make_unique<ClientCache>()) {}
+AIRequestExecutor::~AIRequestExecutor() = default;
+
+Status AIRequestExecutor::execute(const std::string& request_body, std::string& response,
+ const AIResource& config, const AIAdapter& adapter,
+ QueryContext* query_context, bool fail_on_http_error) {
+ auto client = _clients->borrow();
+ // Return the client on every exit path; network I/O holds no cache lock.
+ Defer return_client([&] { _clients->release(std::move(client)); });
+ Status status;
+ for (int64_t attempt = 0; attempt <= config.max_retries; ++attempt) {
+ status = send_request(client.get(), request_body, response, config, adapter, query_context,
+ fail_on_http_error);
+ if (query_context != nullptr && query_context->is_cancelled()) {
+ return query_context->exec_status();
+ }
+ if (status.ok()) {
+ const long http_status = client->get_http_status();
+ // Preserve AI_AGG's immediate rejection of transport-successful non-200.
+ if (http_status != 200) {
+ return Status::HttpError("http status code is not 200, code={}, url={}",
+ http_status, mask_token(config.endpoint));
+ }
+ return status;
+ }
+ if (attempt == config.max_retries) {
Review Comment:
Please classify HTTP failures before retrying, instead of retrying every failed attempt. For example, return immediately for 401/403, retry transient errors such as 408/500/502/503/504, and handle 429 as rate limiting. Please apply the policy consistently to scalar/EMBED and AI_AGG, including the latter's curl fail-on-error path. Repeating the same request with an invalid API key only delays reporting the error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Reuse HTTP connections across AI function requests [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #68816:
URL: https://github.com/apache/doris/pull/68816#discussion_r4253824526
##########
be/src/exprs/function/ai/ai_request_executor.cpp:
##########
@@ -0,0 +1,300 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exprs/function/ai/ai_request_executor.h"
+
+#include <algorithm>
+#include <charconv>
+#include <chrono>
+#include <limits>
+#include <mutex>
+#include <thread>
+#include <vector>
+
+#include "exprs/function/ai/ai_adapter.h"
+#include "runtime/query_context.h"
+#include "service/http/http_client.h"
+#include "util/defer_op.h"
+#include "util/security.h"
+
+namespace doris {
+// Borrowing removes a client from idle, ensuring exclusive use of its easy handle.
+struct AIRequestExecutor::ClientCache {
+ std::unique_ptr<HttpClient> borrow() {
+ {
+ std::lock_guard lock(mutex);
+ if (!idle.empty()) {
+ auto client = std::move(idle.back());
+ idle.pop_back();
+ return client;
+ }
+ }
+ return std::make_unique<HttpClient>();
+ }
+
+ void release(std::unique_ptr<HttpClient> client) {
+ // Bound idle clients, not TCP connections or in-flight requests.
+ // Excess clients are destroyed after releasing the lock.
+ constexpr size_t max_idle_clients = 8;
+ std::lock_guard lock(mutex);
+ if (idle.size() < max_idle_clients) {
+ idle.push_back(std::move(client));
Review Comment:
[P2] Release the copied POST body before caching an idle client. `HttpClient::set_payload` uses `CURLOPT_COPYPOSTFIELDS`, so parking this handle retains the entire last request body until its next `init()` or query teardown. Scalar/EMBED batching permits an over-window input as a singleton, and `ai_context_window_size` has no upper bound; for example, 32 scalar task contexts ending on a 32 MiB LOCAL request can retain about 1 GiB while downstream operators continue. The eight-client limit does not bound these bytes, and [libcurl's malloc copies](https://github.com/curl/curl/blob/curl-8_2_1/lib/setopt.c#L502-L526) are not charged to the query MemTracker. Clear/reset request-owned payload before returning the client to idle; [`curl_easy_reset`](https://curl.se/libcurl/c/curl_easy_reset.html) frees the copied options while preserving live connections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org