You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "lizining1231 (via GitHub)" <gi...@apache.org> on 2026/08/10 20:43:41 UTC
[I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Memory Leak [dubbo-go]
lizining1231 opened a new issue, #3640:
URL: https://github.com/apache/dubbo-go/issues/3640
### Summary
`Server.startHttp2AndHttp3` (`protocol/triple/triple_protocol/server.go`) starts
the HTTP/2 and HTTP/3 servers in parallel with `errgroup.Group{}` but does NOT
hook into the errgroup cancellation mechanism: **as long as either port is
occupied (TCP or UDP)**, the failing side returns an error, while `eg.Wait()`
never returns because it is waiting for the other service that started
normally and is still listening, causing `Run` to hang permanently, and the
successfully-started side is not closed automatically (goroutine/port leak).
### Affected Locations
| Package | Test | Test Location | Related Source Locations |
| --------------------------------- | ---------------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `protocol/triple/triple_protocol` | probe test (temporary, to be promoted to a formal regression test) | `leak_probe_test.go` (local workspace) | `startHttp2AndHttp3` <https://github.com/apache/dubbo-go/blob/master/protocol/triple/triple_protocol/server.go#L288-L308> (`eg := &errgroup.Group{}` does not wire up ctx cancellation) |
### Reproduction
Using the **occupied TCP port** as an example:
```go
// Occupy the TCP port first, then start in dual mode
tcpLn, _ := net.Listen("tcp", tcpAddr) // occupy TCP (HTTP/2 will fail to bind)
srv.Run(constant.CallHTTP2AndHTTP3, tlsConf) // hangs, no return within 3s+
```
Observed output:
```text
leak_probe_test.go:199: Run did not return within 3s: it is hung on errgroup.Wait
leak_probe_test.go:210: UDP port is occupied by HTTP/3 while Run is hung: listen udp 127.0.0.1:43633: bind: address already in use
```
Symmetrically: calling `startHttp2` alone returns the error immediately when
the TCP port is occupied, and calling `startHttp3` alone does the same when
the UDP port is occupied (`listen tcp ...: bind: address already in use`),
proving the failure really happens and is simply not propagated correctly.
<br />
| TCP | UDP | Result |
| -------- | -------- | ----------------------------------------------------- |
| free | free | runs normally |
| free | occupied | HTTP/3 fails + HTTP/2 keeps listening → **Run hangs** |
| occupied | free | HTTP/2 fails + HTTP/3 keeps listening → **Run hangs** |
| occupied | occupied | both fail immediately → `Wait` returns the error |
### Root Cause
- `errgroup.Group.Wait()` waits for **ALL** goroutines to finish before
returning the first error (golang.org/x/sync v0.21.0 errgroup.go:72-102).
- The code uses `&errgroup.Group{}` (not `WithContext`); a failure only
records `g.err` and **never notifies/cancels the other goroutine**.
- Hence after the failing side exits, the successful side is still listening →
`Wait` never returns → `Run` hangs forever; even if the return path were
fixed later, the successful side would not close itself (resource leak).
This chain is **fully symmetric** in both directions: if HTTP/3 fails,
HTTP/2 keeps listening; if HTTP/2 fails, HTTP/3 keeps listening.
### Proposed Fix
Relative to the original implementation (an errgroup launching both
`ListenAndServe*` concurrently), this change:
1. **Add a certificate readiness check**: before startup, verify that
`tlsConf` has a server certificate configured (`Certificates` non-empty, or
`GetCertificate`/`GetConfigForClient` set); return an error immediately if
not — preventing `ServeTLS` from failing synchronously inside a goroutine
and hanging again.
2. **Pre-bind the sockets**: turn the implicit bind into an explicit pre-bind —
`net.Listen("tcp", s.addr)` first, then `net.ListenUDP("udp", s.addr)`;
on either failure, return immediately and close the already-bound socket
(fail-fast, no request has been served at this point).
3. **Serve concurrently**: only after both sockets are bound successfully, run
`ServeTLS(tcpLn, "", "")` and `Serve(udpConn)` concurrently in a plain
`errgroup`, with the result propagated by `eg.Wait()`.
Benefits:
- **Zero new dependencies**: `ServeTLS(ln, "", "")` and
`http3.Server.Serve(conn)` both natively accept pre-bound sockets.
- **Symmetric coverage**: one fix covers both the TCP and UDP error paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Memory Leak [dubbo-go]
Posted by "lizining1231 (via GitHub)" <gi...@apache.org>.
lizining1231 commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5246136057
请问这个issue是否可被采纳？若可以我想得到分配
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak [dubbo-go]
Posted by "Alanxtl (via GitHub)" <gi...@apache.org>.
Alanxtl commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5380782689
close via #3640
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak [dubbo-go]
Posted by "Alanxtl (via GitHub)" <gi...@apache.org>.
Alanxtl closed issue #3640: [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak
URL: https://github.com/apache/dubbo-go/issues/3640
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak [dubbo-go]
Posted by "Alanxtl (via GitHub)" <gi...@apache.org>.
Alanxtl closed issue #3640: [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak
URL: https://github.com/apache/dubbo-go/issues/3640
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak [dubbo-go]
Posted by "lizining1231 (via GitHub)" <gi...@apache.org>.
lizining1231 commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5252199414
好的，谢谢！
标题我已修改为准确的措辞，接下来我会调整方案、注意4个实现细节、添加全面的测试，以高优先级处理该issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Memory Leak [dubbo-go]
Posted by "AlexStocks (via GitHub)" <gi...@apache.org>.
AlexStocks commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5248899342
这个 Issue 可以采纳，按 P1 处理。目前已经分配给你。
我核对了当前 `main`（`48d6e696e755d46df2425bbf71c413bbd1887f3f`）的实现，并分别验证了 TCP 端口被占用和 UDP 端口被占用两种路径：失败侧会立即返回 bind error，另一侧仍继续监听，`errgroup.Wait()` 因等待全部 goroutine 退出而无法返回；调用 `Stop()` 后 `Run()` 才会结束。因此核心问题成立，是双协议启动过程中的生命周期管理和资源清理缺陷。
不过有两点需要调整：
1. 标题中的 “Memory Leak” 建议改为 “Resource Leak” 或 “Startup Hang”。现有证据能确认 listener、端口和 goroutine 被持续占用，但还不足以证明存在不可回收且持续增长的内存泄漏。上层目前是在 goroutine 中调用 `Run()`，所以生产影响也不只是调用方阻塞，还包括服务部分启动、错误未及时反馈以及资源持续占用。
2. Proposed Fix 的“预绑定 TCP/UDP socket”方向可以解决启动阶段的端口冲突，但还不能完整解决生命周期问题。两个 socket 都绑定成功以后，如果任一 `Serve` 异常退出，必须主动关闭另一侧并等待两个 goroutine 都结束；仅改成 `errgroup.WithContext` 不够，因为监听中的 `Serve` 不会仅凭 context 自动退出。
实现时还需要注意：
- 外部创建的 TCP listener 和 UDP `PacketConn` 要明确所有权，并在预绑定失败、任一 Serve 失败、`Stop` 和 `GracefulStop` 路径中保证释放；`http3.Server.Serve(conn)` 不会替调用方关闭传入的连接。
- `net.ListenUDP` 的第二个参数是 `*net.UDPAddr`，不能直接传 `s.addr` 字符串；可以使用 `net.ResolveUDPAddr` + `net.ListenUDP`，或者 `net.ListenPacket`。
- TLS readiness 检查需要同时支持 `Certificates`、`GetCertificate` 和 `GetConfigForClient`。
- 请补充以下回归测试：TCP 被占用、UDP 被占用、TLS 配置没有可用证书、绑定成功后任一 Serve 异常退出、`Stop`/`GracefulStop` 后 TCP 和 UDP 端口均可重新绑定，以及相关 race 测试。
另外，#3639 正在修改同一个 Server 的生命周期字段访问，但它只处理并发访问问题，不会修复本 Issue；实现时需要协调或 rebase，避免覆盖其中的 atomic pointer 修改。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Resource Leak [dubbo-go]
Posted by "Alanxtl (via GitHub)" <gi...@apache.org>.
Alanxtl commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5253205525
close via #3639
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [I] [Bug] HTTP/2+HTTP/3 Dual-Protocol Startup Failure Causes Run to Hang Forever with Memory Leak [dubbo-go]
Posted by "lizining1231 (via GitHub)" <gi...@apache.org>.
lizining1231 commented on issue #3640:
URL: https://github.com/apache/dubbo-go/issues/3640#issuecomment-5245971865
我尝试构造了一份 最小复现测试 用于辅助该issue，它在当前场景下稳定报错
1. 测试的报错日志是：
```text
=== RUN TestRepro_RunHangsOnOccupiedTCP
repro_issue9_test.go:125: BUG REPRODUCED: Run did not return within 3s after HTTP/2 failed to start (hung on errgroup.Wait)
--- FAIL: TestRepro_RunHangsOnOccupiedTCP (3.00s)
```
2. 测试的行为是：
测试拿到空闲 TCP 端口并验证同端口 UDP 空闲，然后占住 TCP 端口，用这个地址创建开启 HTTP/3 的 Server，在 goroutine 里调用 srv.Run 并把结果发到 channel。主 goroutine 用 select 等 3 秒：收到错误就打 t.Logf 日志，超时则打 t.Errorf 的 BUG REPRODUCED 日志
3. 测试的预期是：
如果 Run 在这 3 秒内返回了，带着 HTTP/2 的 bind 错误及时返回，测试通过。反之则FAIL，bug复现
4. 测试的原文件是：
[repro_issue3640_test.txt](https://github.com/user-attachments/files/30913881/repro_issue3640_test.txt)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org