You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "awickham10 (via GitHub)" <gi...@apache.org> on 2023/06/30 15:54:45 UTC

[GitHub] [incubator-devlake] awickham10 opened a new issue, #5612: [Bug] Go Panic in OpenShift Kubernetes

awickham10 opened a new issue, #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   I have deployed the devlake v0.17.0-beta12 helm chart to an OpenShift Kubernetes environment. When I try to add any sort of connection, the lake pod will crash with a go panic.
   
   ### What do you expect to happen
   
   Panic should not happen and can add connections.
   
   ### How to reproduce
   
   Deploy the  helm chart to an OpenShift Kubernetes environment and then try to add a webhook connection.
   
   ### Anything else
   
   This happens very frequently. Below is a copy of the panic.
   
   ```
   [GIN] 2023/06/30 - 15:51:41 | 200 |       2.713µs |     10.242.18.2 | GET      "/ping"
   [GIN] 2023/06/30 - 15:51:48 | 200 |     3.71345ms |  10.128.160.136 | GET      "/plugins/webhook/connections"
   [GIN] 2023/06/30 - 15:51:48 | 200 |     855.468µs |  10.128.160.136 | GET      "/plugins/webhook/connections"
   [GIN] 2023/06/30 - 15:51:51 | 200 |        2.76µs |     10.242.18.2 | GET      "/ping"
   runtime: g 2787: unexpected return pc for github.com/gin-gonic/gin.(*Engine).handleHTTPRequest called from 0x1cc50
   stack: frame={sp:0xc0019c5900, fp:0xc0019c5a88} stack=[0xc0019c2000,0xc0019c6000)
   0x000000c0019c5800:  0x000000c0019d78f0  0x00000c6af8bd00e2 
   0x000000c0019c5810:  0xc11fdc65e8825faa  0x0000000000000000 
   0x000000c0019c5820:  0x0000000000000000  0x000000c000512035 
   0x000000c0019c5830:  0x000000c0001a0008  0x0000000000000000 
   0x000000c0019c5840:  0x000000000203b2f0  0x0000000000000000 
   0x000000c0019c5850:  0x000000000412b080  0x000000c0019d78d0 
   0x000000c0019c5860:  0x0000000000000013  0x0000000000000000 
   0x000000c0019c5870:  0x000000000000000b  0x000000c000512046 
   0x000000c0019c5880:  0x000000c00010f1b1  0x0000000000000008 
   0x000000c0019c5890:  0x00007f5e3f7bca08  0x0000000000000014 
   0x000000c0019c58a0:  0x0000000000000000  0x000000c00196cee0 
   0x000000c0019c58b0:  0x000000c000512046  0x000000c00195f0b0 
   0x000000c0019c58c0:  0x0000000000000005  0x0000000000000005 
   0x000000c0019c58d0:  0x0000000000000000  0x0000000000000000 
   0x000000c0019c58e0:  0x000000c00010f1a0  0x000000000000001c 
   0x000000c0019c58f0:  0x000000c0019c5a78  0x000000000129f411 <github.com/gin-gonic/gin.(*Engine).handleHTTPRequest+0x0000000000000671> 
   0x000000c0019c5900: <0x000000c0001faa00  0x000000c000512035 
   0x000000c0019c5910:  0x0000000000abc05f <runtime.mallocgc+0x000000000000069f>  0x000000c00019ef78 
   0x000000c0019c5920:  0x000000c00019ef90  0x0000000000abc000 <runtime.mallocgc+0x0000000000000640> 
   0x000000c0019c5930:  0x0000000000abc05f <runtime.mallocgc+0x000000000000069f>  0x0000000000000005 
   0x000000c0019c5940:  0x000000000000001c  0x0000000000000001 
   0x000000c0019c5950:  0x0000000000000004  0x000000c0019d7990 
   0x000000c0019c5960:  0x0000000000000003  0x000000c0001d0460 
   0x000000c0019c5970:  0x000000c0000f1ce0  0x000000c000512035 
   0x000000c0019c5980:  0x000000c000512030  0x000000c0003af800 
   0x000000c0019c5990:  0x000000c0019d79e0  0x0000000000f39088 <net/http.newBufioWriterSize+0x0000000000000068> 
   0x000000c0019c59a0:  0x000000000412a4e0  0x0000000000000000 
   0x000000c0019c59b0:  0x000000c0019d79e0  0x000000c0019d79e0 
   0x000000c0019c59c0:  0x0000000000abc3a7 <runtime.newobject+0x0000000000000027>  0x00000000000000e0 
   0x000000c0019c59d0:  0x000000c00195f0b0  0x0000000000000005 
   0x000000c0019c59e0:  0x0000000000000005  0x0000000000000000 
   0x000000c0019c59f0:  0x000000000001ce00  0x000000c00010f1a0 
   0x000000c0019c5a00:  0x000000000000001c  0x0000000000000000 
   0x000000c0019c5a10:  0x0000000000000000  0x0000000000000000 
   0x000000c0019c5a20:  0x0000000100052c40  0x0000000000b188b6 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c0019c5a30:  0x000000000001caf1  0x00007f5e47d61cc0 
   0x000000c0019c5a40:  0x000000c00195f0b0  0x0000000000000005 
   0x000000c0019c5a50:  0x0000000000000005  0x0000000000000000 
   0x000000c0019c5a60:  0x000000000001ce00  0x000000c00010f1a0 
   0x000000c0019c5a70:  0x000000000000001c  0x000000c001ac1ab8 
   0x000000c0019c5a80: !0x000000000001cc50 >0x000000c000600d00 
   0x000000c0019c5a90:  0x000000c0001faa00  0x0000000000000000 
   0x000000c0019c5aa0:  0x000000c0001faa00  0x000000c000600e18 
   0x000000c0019c5ab0:  0x00000000023ccae0  0x000000c001ac1b68 
   0x000000c0019c5ac0:  0x0000000000f43c2c <net/http.serverHandler.ServeHTTP+0x000000000000030c>  0x000000c000600d00 
   0x000000c0019c5ad0:  0x00000000023ccae0  0x000000c0019ee380 
   0x000000c0019c5ae0:  0x000000c001aec100  0x0000000000b25dae <sync.(*Pool).Get+0x000000000000008e> 
   0x000000c0019c5af0:  0x000100000412a520  0x0000000000000000 
   0x000000c0019c5b00:  0x00007f5e3f7e2d38  0x00000000023bba00 
   0x000000c0019c5b10:  0x000000c000052cec  0x0000000000b188b6 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c0019c5b20:  0x000000000001a511  0x000000c000600d00 
   0x000000c0019c5b30:  0x000000c0019d7b40  0x000000c0000deb40 
   0x000000c0019c5b40:  0x000000000001a930  0x000000c0003741a0 
   0x000000c0019c5b50:  0x000000000001a8d7  0x0000000000b18880 <runtime.asmcgocall+0x0000000000000000> 
   0x000000c0019c5b60:  0x0000000000000000  0x000000c001ac1fa8 
   0x000000c0019c5b70:  0x000000000001a670  0x00000000023c3fb8 
   0x000000c0019c5b80:  0x00000000023ccae0 
   fatal error: unknown caller pc
   
   runtime stack:
   runtime.throw({0x7f5da5f6cd78?, 0x2ec8000?})
   	/usr/local/go/src/runtime/panic.go:1047 +0x5f fp=0x7f5e21ffa758 sp=0x7f5e21ffa728 pc=0x7f5da5870e9f
   runtime.gentraceback(0x7f5e47b91628?, 0xc000535734?, 0x0?, 0xc000535734?, 0x0, 0x0, 0x7fffffff, 0x203e150, 0x0?, 0x0)
   	/usr/local/go/src/runtime/traceback.go:269 +0x1c8a fp=0x7f5e21ffaac0 sp=0x7f5e21ffa758 pc=0x7f5da589356a
   runtime.copystack(0xc0003741a0, 0x800000002?)
   	/usr/local/go/src/runtime/stack.go:932 +0x30d fp=0x7f5e21ffac78 sp=0x7f5e21ffaac0 pc=0x7f5da5888a2d
   runtime.newstack()
   	/usr/local/go/src/runtime/stack.go:1112 +0x498 fp=0x7f5e21ffae30 sp=0x7f5e21ffac78 pc=0x7f5da5888fb8
   runtime.morestack()
   	/usr/local/go/src/runtime/asm_amd64.s:570 +0x80 fp=0x7f5e21ffae38 sp=0x7f5e21ffae30 pc=0x7f5da589b000
   
   goroutine 2787 [copystack]:
   runtime.heapBitsSetType(0xc000012438?, 0x18?, 0x18?, 0x1f05c20?)
   	/usr/local/go/src/runtime/mbitmap.go:844 +0xc16 fp=0xc0019c4390 sp=0xc0019c4388 pc=0x7f5da5850416
   runtime.mallocgc(0x18, 0x1f05c20, 0x1)
   	/usr/local/go/src/runtime/malloc.go:1050 +0x68e fp=0xc0019c4408 sp=0xc0019c4390 pc=0x7f5da584730e
   runtime.makeslice(0xc00057a380?, 0xc00057a4d0?, 0xc00057a540?)
   	/usr/local/go/src/runtime/slice.go:103 +0x52 fp=0xc0019c4430 sp=0xc0019c4408 pc=0x7f5da5886992
   reflect.Value.MapKeys({0x1d09520?, 0xc00198ccf0?, 0xc00054f210?})
   	/usr/local/go/src/reflect/value.go:1702 +0x105 fp=0xc0019c4508 sp=0xc0019c4430 pc=0x7f5da58f4025
   github.com/mitchellh/mapstructure.(*Decoder).decodeStructFromMap(0xc0001a00b8, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?, 0x7f5e3ebf8200?}, {0x7f5da617ece0?, 0xc000180690?, 0x7f5e3ebf8200?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1243 +0x27d fp=0xc0019c4d50 sp=0xc0019c4508 pc=0x7f5da5d8405d
   github.com/mitchellh/mapstructure.(*Decoder).decodeStruct(0x0?, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?}, {0x7f5da617ece0?, 0xc000180690?, 0x300?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1203 +0x5e8 fp=0xc0019c4e88 sp=0xc0019c4d50 pc=0x7f5da5d83d28
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc0001a00b8, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?}, {0x7f5da617ece0?, 0xc000180690?, 0x7f5da5899959?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:454 +0x329 fp=0xc0019c4f58 sp=0xc0019c4e88 pc=0x7f5da5d7abc9
   github.com/mitchellh/mapstructure.(*Decoder).decodePtr(0xc0001a00b8, {0x0, 0x0}, {0x1d09520, 0xc00198ccf0}, {0x7f5da61357e0?, 0xc0001a00e8?, 0x7f5da584735f?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1020 +0x44a fp=0xc0019c4ff0 sp=0xc0019c4f58 pc=0x7f5da5d815ca
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc0001a00b8, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?}, {0x7f5da61357e0?, 0xc0001a00e8?, 0xc0001a00e0?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:458 +0x2e9 fp=0xc0019c50c0 sp=0xc0019c4ff0 pc=0x7f5da5d7ab89
   github.com/mitchellh/mapstructure.(*Decoder).decodePtr(0xc0001a00b8, {0x0, 0x0}, {0x1d09520, 0xc00198ccf0}, {0xc000052dc0?, 0xc0001a00e8?, 0x7f5e44076108?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1026 +0x4ea fp=0xc0019c5158 sp=0xc0019c50c0 pc=0x7f5da5d8166a
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc0001a00b8, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?}, {0xc000052dc0?, 0xc0001a00e8?, 0xc00037a640?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:458 +0x2e9 fp=0xc0019c5228 sp=0xc0019c5158 pc=0x7f5da5d7ab89
   github.com/mitchellh/mapstructure.(*Decoder).decodeBasic(0x1cdbe60?, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0}, {0x1cdbe60?, 0xc00037a650?, 0xc000052d40?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:504 +0x66c fp=0xc0019c5380 sp=0xc0019c5228 pc=0x7f5da5d7b72c
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc0001a00b8, {0x0, 0x0}, {0x1d09520?, 0xc00198ccf0?}, {0x1cdbe60?, 0xc00037a650?, 0x40?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:444 +0x2c5 fp=0xc0019c5450 sp=0xc0019c5380 pc=0x7f5da5d7ab65
   github.com/mitchellh/mapstructure.(*Decoder).Decode(0xc0001a00b8, {0x1d09520, 0xc00198ccf0})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:389 +0xea fp=0xc0019c54b0 sp=0xc0019c5450 pc=0x7f5da5d7a86a
   github.com/mitchellh/mapstructure.Decode({0x1d09520, 0xc00198ccf0}, {0x1c3ad60?, 0xc00037a650})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:297 +0x95 fp=0xc0019c54e0 sp=0xc0019c54b0 pc=0x7f5da5d7a495
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.Decode({0x1d09520, 0xc00198ccf0}, {0x7f5da61357e0, 0xc000180690}, 0xc000668240)
   	/app/helpers/pluginhelper/api/mapstructure.go:86 +0xc6 fp=0xc0019c5538 sp=0xc0019c54e0 pc=0x7f5da5f41a86
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*ConnectionApiHelper).merge(0xc0003ae000, {0x7f5da61357e0?, 0xc000180690?}, 0x7f5da58476c7?)
   	/app/helpers/pluginhelper/api/connection_helper.go:116 +0xe5 fp=0xc0019c5598 sp=0xc0019c5538 pc=0x7f5da5f40665
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*ConnectionApiHelper).Create(0xc0003ae000, {0x7f5da61357e0, 0xc000180690}, 0xc000628230?)
   	/app/helpers/pluginhelper/api/connection_helper.go:58 +0x30 fp=0xc0019c55e0 sp=0xc0019c5598 pc=0x7f5da5f3ff70
   github.com/apache/incubator-devlake/plugins/webhook/api.PostConnections(0xc00198cc30?)
   	/app/plugins/webhook/api/connection.go:41 +0x52 fp=0xc0019c5620 sp=0xc0019c55e0 pc=0x7f5da5f43652
   github.com/apache/incubator-devlake/server/api.handlePluginCall.func1(0xc0001faa00)
   	/app/server/api/router.go:119 +0x1fe fp=0xc0019c56f8 sp=0xc0019c5620 pc=0x1b965de
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0001faa00)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/recovery.go:99 +0x82 fp=0xc0019c5748 sp=0xc0019c56f8 pc=0x12a13e2
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0001faa00)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/logger.go:241 +0xe7 fp=0xc0019c5900 sp=0xc0019c5748 pc=0x12a0527
   runtime: g 2787: unexpected return pc for github.com/gin-gonic/gin.(*Engine).handleHTTPRequest called from 0x1cc50
   stack: frame={sp:0xc0019c5900, fp:0xc0019c5a88} stack=[0xc0019c2000,0xc0019c6000)
   0x000000c0019c5800:  0x000000c0019d78f0  0x00000c6af8bd00e2 
   0x000000c0019c5810:  0xc11fdc65e8825faa  0x0000000000000000 
   0x000000c0019c5820:  0x0000000000000000  0x000000c000512035 
   0x000000c0019c5830:  0x000000c0001a0008  0x0000000000000000 
   0x000000c0019c5840:  0x000000000203b2f0  0x0000000000000000 
   0x000000c0019c5850:  0x000000000412b080  0x000000c0019d78d0 
   0x000000c0019c5860:  0x0000000000000013  0x0000000000000000 
   0x000000c0019c5870:  0x000000000000000b  0x000000c000512046 
   0x000000c0019c5880:  0x000000c00010f1b1  0x0000000000000008 
   0x000000c0019c5890:  0x00007f5e3f7bca08  0x0000000000000014 
   0x000000c0019c58a0:  0x0000000000000000  0x000000c00196cee0 
   0x000000c0019c58b0:  0x000000c000512046  0x000000c00195f0b0 
   0x000000c0019c58c0:  0x0000000000000005  0x0000000000000005 
   0x000000c0019c58d0:  0x0000000000000000  0x0000000000000000 
   0x000000c0019c58e0:  0x000000c00010f1a0  0x000000000000001c 
   0x000000c0019c58f0:  0x000000c0019c5a78  0x000000000129f411 <github.com/gin-gonic/gin.(*Engine).handleHTTPRequest+0x0000000000000671> 
   0x000000c0019c5900: <0x000000c0001faa00  0x000000c000512035 
   0x000000c0019c5910:  0x0000000000abc05f <runtime.mallocgc+0x000000000000069f>  0x000000c00019ef78 
   0x000000c0019c5920:  0x000000c00019ef90  0x0000000000abc000 <runtime.mallocgc+0x0000000000000640> 
   0x000000c0019c5930:  0x0000000000abc05f <runtime.mallocgc+0x000000000000069f>  0x0000000000000005 
   0x000000c0019c5940:  0x000000000000001c  0x0000000000000001 
   0x000000c0019c5950:  0x0000000000000004  0x000000c0019d7990 
   0x000000c0019c5960:  0x0000000000000003  0x000000c0001d0460 
   0x000000c0019c5970:  0x000000c0000f1ce0  0x000000c000512035 
   0x000000c0019c5980:  0x000000c000512030  0x000000c0003af800 
   0x000000c0019c5990:  0x000000c0019d79e0  0x0000000000f39088 <net/http.newBufioWriterSize+0x0000000000000068> 
   0x000000c0019c59a0:  0x000000000412a4e0  0x0000000000000000 
   0x000000c0019c59b0:  0x000000c0019d79e0  0x000000c0019d79e0 
   0x000000c0019c59c0:  0x0000000000abc3a7 <runtime.newobject+0x0000000000000027>  0x00000000000000e0 
   0x000000c0019c59d0:  0x000000c00195f0b0  0x0000000000000005 
   0x000000c0019c59e0:  0x0000000000000005  0x0000000000000000 
   0x000000c0019c59f0:  0x000000000001ce00  0x000000c00010f1a0 
   0x000000c0019c5a00:  0x000000000000001c  0x0000000000000000 
   0x000000c0019c5a10:  0x0000000000000000  0x0000000000000000 
   0x000000c0019c5a20:  0x0000000100052c40  0x0000000000b188b6 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c0019c5a30:  0x000000000001caf1  0x00007f5e47d61cc0 
   0x000000c0019c5a40:  0x000000c00195f0b0  0x0000000000000005 
   0x000000c0019c5a50:  0x0000000000000005  0x0000000000000000 
   0x000000c0019c5a60:  0x000000000001ce00  0x000000c00010f1a0 
   0x000000c0019c5a70:  0x000000000000001c  0x000000c001ac1ab8 
   0x000000c0019c5a80: !0x000000000001cc50 >0x000000c000600d00 
   0x000000c0019c5a90:  0x000000c0001faa00  0x0000000000000000 
   0x000000c0019c5aa0:  0x000000c0001faa00  0x000000c000600e18 
   0x000000c0019c5ab0:  0x00000000023ccae0  0x000000c001ac1b68 
   0x000000c0019c5ac0:  0x0000000000f43c2c <net/http.serverHandler.ServeHTTP+0x000000000000030c>  0x000000c000600d00 
   0x000000c0019c5ad0:  0x00000000023ccae0  0x000000c0019ee380 
   0x000000c0019c5ae0:  0x000000c001aec100  0x0000000000b25dae <sync.(*Pool).Get+0x000000000000008e> 
   0x000000c0019c5af0:  0x000100000412a520  0x0000000000000000 
   0x000000c0019c5b00:  0x00007f5e3f7e2d38  0x00000000023bba00 
   0x000000c0019c5b10:  0x000000c000052cec  0x0000000000b188b6 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c0019c5b20:  0x000000000001a511  0x000000c000600d00 
   0x000000c0019c5b30:  0x000000c0019d7b40  0x000000c0000deb40 
   0x000000c0019c5b40:  0x000000000001a930  0x000000c0003741a0 
   0x000000c0019c5b50:  0x000000000001a8d7  0x0000000000b18880 <runtime.asmcgocall+0x0000000000000000> 
   0x000000c0019c5b60:  0x0000000000000000  0x000000c001ac1fa8 
   0x000000c0019c5b70:  0x000000000001a670  0x00000000023c3fb8 
   0x000000c0019c5b80:  0x00000000023ccae0 
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000600d00, 0xc0001faa00)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/gin.go:555 +0x671 fp=0xc0019c5a88 sp=0xc0019c5900 pc=0x129f411
   
   goroutine 1 [IO wait]:
   runtime.gopark(0x200000003?, 0xc0000061a0?, 0x0?, 0x65?, 0xc0001499d0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000149960 sp=0xc000149940 pc=0xae9996
   runtime.netpollblock(0xc0001499b0?, 0x149a28?, 0xc0?)
   	/usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc000149998 sp=0xc000149960 pc=0xae1d77
   internal/poll.runtime_pollWait(0x7f5e3f78f738, 0x72)
   	/usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc0001499b8 sp=0xc000149998 pc=0xb12909
   internal/poll.(*pollDesc).wait(0xc001945e80?, 0x6?, 0x0)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0001499e0 sp=0xc0001499b8 pc=0xb91372
   internal/poll.(*pollDesc).waitRead(...)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Accept(0xc001945e80)
   	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x234 fp=0xc000149a78 sp=0xc0001499e0 pc=0xb96774
   net.(*netFD).accept(0xc001945e80)
   	/usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc000149b30 sp=0xc000149a78 pc=0xe21955
   net.(*TCPListener).accept(0xc00019eba0)
   	/usr/local/go/src/net/tcpsock_posix.go:142 +0x28 fp=0xc000149b60 sp=0xc000149b30 pc=0xe3e468
   net.(*TCPListener).Accept(0xc00019eba0)
   	/usr/local/go/src/net/tcpsock.go:288 +0x3d fp=0xc000149b90 sp=0xc000149b60 pc=0xe3d4dd
   net/http.(*onceCloseListener).Accept(0xc00065aa00?)
   	<autogenerated>:1 +0x2a fp=0xc000149ba8 sp=0xc000149b90 pc=0xf6ceca
   net/http.(*Server).Serve(0xc0000deb40, {0x23cc8d0, 0xc00019eba0})
   	/usr/local/go/src/net/http/server.go:3070 +0x385 fp=0xc000149cd8 sp=0xc000149ba8 pc=0xf44265
   net/http.(*Server).ListenAndServe(0xc0000deb40)
   	/usr/local/go/src/net/http/server.go:2999 +0x7d fp=0xc000149d08 sp=0xc000149cd8 pc=0xf43dbd
   net/http.ListenAndServe(...)
   	/usr/local/go/src/net/http/server.go:3255
   github.com/gin-gonic/gin.(*Engine).Run(0xc000600d00, {0xc003507e68, 0x1, 0x1})
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/gin.go:356 +0x1d8 fp=0xc000149d98 sp=0xc000149d08 pc=0x129d4b8
   github.com/apache/incubator-devlake/server/api.CreateApiService()
   	/app/server/api/api.go:154 +0x665 fp=0xc000149f40 sp=0xc000149d98 pc=0x1b95545
   main.main()
   	/app/server/main.go:39 +0x7e fp=0xc000149f80 sp=0xc000149f40 pc=0x1b9693e
   runtime.main()
   	/usr/local/go/src/runtime/proc.go:250 +0x212 fp=0xc000149fe0 sp=0xc000149f80 pc=0xae95d2
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000149fe8 sp=0xc000149fe0 pc=0xb18c21
   
   goroutine 2 [force gc (idle), 2 minutes]:
   runtime.gopark(0x2b7af9d6e03f7b?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000074fb0 sp=0xc000074f90 pc=0xae9996
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.forcegchelper()
   	/usr/local/go/src/runtime/proc.go:302 +0xad fp=0xc000074fe0 sp=0xc000074fb0 pc=0xae982d
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0xb18c21
   
   goroutine 3 [GC sweep wait]:
   runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000075790 sp=0xc000075770 pc=0xae9996
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.bgsweep(0x0?)
   	/usr/local/go/src/runtime/mgcsweep.go:297 +0xd7 fp=0xc0000757c8 sp=0xc000075790 pc=0xad4457
   runtime.gcenable.func1()
   	/usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000757e0 sp=0xc0000757c8 pc=0xac8fe6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000757e8 sp=0xc0000757e0 pc=0xb18c21
   
   goroutine 4 [GC scavenge wait]:
   runtime.gopark(0x412b100?, 0x1b2730?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000075f70 sp=0xc000075f50 pc=0xae9996
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.(*scavengerState).park(0x412b100)
   	/usr/local/go/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc000075fa0 sp=0xc000075f70 pc=0xad2433
   runtime.bgscavenge(0x0?)
   	/usr/local/go/src/runtime/mgcscavenge.go:622 +0x65 fp=0xc000075fc8 sp=0xc000075fa0 pc=0xad2a45
   runtime.gcenable.func2()
   	/usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000075fe0 sp=0xc000075fc8 pc=0xac8f86
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0xb18c21
   
   goroutine 18 [finalizer wait, 227 minutes]:
   runtime.gopark(0x412c1c0?, 0xc0001861a0?, 0x0?, 0x0?, 0xc000074770?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000074628 sp=0xc000074608 pc=0xae9996
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.runfinq()
   	/usr/local/go/src/runtime/mfinal.go:180 +0x10f fp=0xc0000747e0 sp=0xc000074628 pc=0xac80ef
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000747e8 sp=0xc0000747e0 pc=0xb18c21
   
   goroutine 19 [semacquire, 227 minutes]:
   runtime.gopark(0xc0000706b8?, 0xafa895?, 0x0?, 0x23?, 0xc0002c0000?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000070668 sp=0xc000070648 pc=0xae9996
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.semacquire1(0xc00029ec48, 0x0?, 0x1, 0x0)
   	/usr/local/go/src/runtime/sema.go:150 +0x1fe fp=0xc0000706d0 sp=0xc000070668 pc=0xafa5de
   sync.runtime_Semacquire(0x1025580?)
   	/usr/local/go/src/runtime/sema.go:62 +0x25 fp=0xc000070700 sp=0xc0000706d0 pc=0xb14ee5
   sync.(*WaitGroup).Wait(0xc0002be0a0?)
   	/usr/local/go/src/sync/waitgroup.go:139 +0x52 fp=0xc000070728 sp=0xc000070700 pc=0xb27272
   github.com/spf13/viper.(*Viper).WatchConfig.func1()
   	/go/pkg/mod/github.com/spf13/viper@v1.8.1/viper.go:410 +0x305 fp=0xc0000707e0 sp=0xc000070728 pc=0x10bf0a5
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000707e8 sp=0xc0000707e0 pc=0xb18c21
   
   goroutine 20 [syscall, 227 minutes]:
   syscall.Syscall6(0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/syscall/syscall_linux.go:90 +0x36 fp=0xc00030fb18 sp=0xc00030fa90 pc=0xb31436
   syscall.Syscall6(0xe8, 0x10, 0xc00030fc14, 0x7, 0xffffffffffffffff, 0x0, 0x0)
   	<autogenerated>:1 +0x45 fp=0xc00030fb60 sp=0xc00030fb18 pc=0xb317e5
   golang.org/x/sys/unix.EpollWait(0x0?, {0xc00030fc14?, 0x0?, 0x0?}, 0x0?)
   	/go/pkg/mod/golang.org/x/sys@v0.7.0/unix/zsyscall_linux_amd64.go:56 +0x58 fp=0xc00030fbd0 sp=0xc00030fb60 pc=0xf92b98
   github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc0002ae540)
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify_poller.go:87 +0x7d fp=0xc00030fc80 sp=0xc00030fbd0 pc=0x102687d
   github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc0002be0a0)
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify.go:193 +0x275 fp=0xc00031ffc8 sp=0xc00030fc80 pc=0x1025b55
   github.com/fsnotify/fsnotify.NewWatcher.func1()
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify.go:60 +0x26 fp=0xc00031ffe0 sp=0xc00031ffc8 pc=0x1025126
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00031ffe8 sp=0xc00031ffe0 pc=0xb18c21
   
   goroutine 21 [select, 227 minutes]:
   runtime.gopark(0xc0000717b0?, 0x2?, 0x0?, 0x0?, 0xc000071734?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000324d98 sp=0xc000324d78 pc=0xae9996
   runtime.selectgo(0xc000324fb0, 0xc000071730, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000324ed8 sp=0xc000324d98 pc=0xaf935c
   github.com/spf13/viper.(*Viper).WatchConfig.func1.1()
   	/go/pkg/mod/github.com/spf13/viper@v1.8.1/viper.go:371 +0x105 fp=0xc000324fe0 sp=0xc000324ed8 pc=0x10bf205
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000324fe8 sp=0xc000324fe0 pc=0xb18c21
   
   goroutine 22 [GC worker (idle), 227 minutes]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000071f50 sp=0xc000071f30 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000071fe0 sp=0xc000071f50 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0xb18c21
   
   goroutine 23 [GC worker (idle), 227 minutes]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000072750 sp=0xc000072730 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000727e0 sp=0xc000072750 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0xb18c21
   
   goroutine 34 [GC worker (idle), 227 minutes]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000606750 sp=0xc000606730 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0006067e0 sp=0xc000606750 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0006067e8 sp=0xc0006067e0 pc=0xb18c21
   
   goroutine 35 [GC worker (idle), 227 minutes]:
   runtime.gopark(0x2b6e992d1ebf9f?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000606f50 sp=0xc000606f30 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000606fe0 sp=0xc000606f50 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000606fe8 sp=0xc000606fe0 pc=0xb18c21
   
   goroutine 50 [GC worker (idle), 227 minutes]:
   runtime.gopark(0x2b6e992d2a7be2?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000602750 sp=0xc000602730 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0006027e0 sp=0xc000602750 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0006027e8 sp=0xc0006027e0 pc=0xb18c21
   
   goroutine 24 [GC worker (idle)]:
   runtime.gopark(0x2b7af9d793a809?, 0x1?, 0x23?, 0xda?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000072f50 sp=0xc000072f30 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000072fe0 sp=0xc000072f50 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0xb18c21
   
   goroutine 25 [GC worker (idle)]:
   runtime.gopark(0x2b7af9d7939d80?, 0x1?, 0xe1?, 0x8c?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000073750 sp=0xc000073730 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000737e0 sp=0xc000073750 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0xb18c21
   
   goroutine 36 [GC worker (idle), 4 minutes]:
   runtime.gopark(0x2b7ac1cee8b385?, 0x1?, 0x0?, 0x42?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000607750 sp=0xc000607730 pc=0xae9996
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0006077e0 sp=0xc000607750 pc=0xacb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0006077e8 sp=0xc0006077e0 pc=0xb18c21
   
   goroutine 51 [chan receive]:
   runtime.gopark(0xc0000a9b00?, 0x2020202020205445?, 0x0?, 0x0?, 0xca00243ca00e9d7c?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000604e98 sp=0xc000604e78 pc=0xae9996
   runtime.chanrecv(0xc0001a2360, 0xc000604f98, 0x1)
   	/usr/local/go/src/runtime/chan.go:583 +0x49b fp=0xc000604f28 sp=0xc000604e98 pc=0xab54fb
   runtime.chanrecv2(0xc0001a9578?, 0x3b9aca00?)
   	/usr/local/go/src/runtime/chan.go:447 +0x18 fp=0xc000604f50 sp=0xc000604f28 pc=0xab5038
   github.com/panjf2000/ants/v2.(*Pool).purgePeriodically(0xc0004e8b60)
   	/go/pkg/mod/github.com/panjf2000/ants/v2@v2.4.6/pool.go:69 +0x8b fp=0xc000604fc8 sp=0xc000604f50 pc=0x1691b6b
   github.com/panjf2000/ants/v2.NewPool.func2()
   	/go/pkg/mod/github.com/panjf2000/ants/v2@v2.4.6/pool.go:137 +0x26 fp=0xc000604fe0 sp=0xc000604fc8 pc=0x1692106
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000604fe8 sp=0xc000604fe0 pc=0xb18c21
   
   goroutine 52 [select, 227 minutes]:
   runtime.gopark(0xc000605788?, 0x2?, 0x11?, 0x0?, 0xc000605784?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000605610 sp=0xc0006055f0 pc=0xae9996
   runtime.selectgo(0xc000605788, 0xc000605780, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000605750 sp=0xc000605610 pc=0xaf935c
   database/sql.(*DB).connectionOpener(0xc001906410, {0x23cdfd0, 0xc0001ef800})
   	/usr/local/go/src/database/sql/sql.go:1224 +0x8d fp=0xc0006057b8 sp=0xc000605750 pc=0x111930d
   database/sql.OpenDB.func1()
   	/usr/local/go/src/database/sql/sql.go:792 +0x2e fp=0xc0006057e0 sp=0xc0006057b8 pc=0x11175ce
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0006057e8 sp=0xc0006057e0 pc=0xb18c21
   
   goroutine 26 [select, 48 minutes]:
   runtime.gopark(0xc000085f90?, 0x2?, 0x0?, 0x20?, 0xc000085f74?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000085de8 sp=0xc000085dc8 pc=0xae9996
   runtime.selectgo(0xc000085f90, 0xc000085f70, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000085f28 sp=0xc000085de8 pc=0xaf935c
   database/sql.(*DB).connectionCleaner(0xc001906410, 0x1?)
   	/usr/local/go/src/database/sql/sql.go:1067 +0xa7 fp=0xc000085fc0 sp=0xc000085f28 pc=0x11186e7
   database/sql.(*DB).startCleanerLocked.func1()
   	/usr/local/go/src/database/sql/sql.go:1054 +0x2a fp=0xc000085fe0 sp=0xc000085fc0 pc=0x111860a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0xb18c21
   
   goroutine 27 [select, 227 minutes]:
   runtime.gopark(0xc000605f88?, 0x2?, 0xd0?, 0xdf?, 0xc000605f84?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000605e10 sp=0xc000605df0 pc=0xae9996
   runtime.selectgo(0xc000605f88, 0xc000605f80, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000605f50 sp=0xc000605e10 pc=0xaf935c
   database/sql.(*DB).connectionOpener(0xc0005956c0, {0x23cdfd0, 0xc0003af000})
   	/usr/local/go/src/database/sql/sql.go:1224 +0x8d fp=0xc000605fb8 sp=0xc000605f50 pc=0x111930d
   database/sql.OpenDB.func1()
   	/usr/local/go/src/database/sql/sql.go:792 +0x2e fp=0xc000605fe0 sp=0xc000605fb8 pc=0x11175ce
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000605fe8 sp=0xc000605fe0 pc=0xb18c21
   
   goroutine 67 [select, 227 minutes]:
   runtime.gopark(0xc001935f80?, 0x2?, 0xd0?, 0xdf?, 0xc001935f40?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000086dc0 sp=0xc000086da0 pc=0xae9996
   runtime.selectgo(0xc000086f80, 0xc001935f3c, 0xc001935f70?, 0x0, 0xc0019064e0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000086f00 sp=0xc000086dc0 pc=0xaf935c
   github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1()
   	/go/pkg/mod/github.com/go-sql-driver/mysql@v1.6.0/connection.go:614 +0xaa fp=0xc000086fe0 sp=0xc000086f00 pc=0x16e330a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000086fe8 sp=0xc000086fe0 pc=0xb18c21
   
   goroutine 68 [select, 48 minutes]:
   runtime.gopark(0xc000087f90?, 0x2?, 0x0?, 0x20?, 0xc000087f74?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000087de8 sp=0xc000087dc8 pc=0xae9996
   runtime.selectgo(0xc000087f90, 0xc000087f70, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000087f28 sp=0xc000087de8 pc=0xaf935c
   database/sql.(*DB).connectionCleaner(0xc0005956c0, 0x1?)
   	/usr/local/go/src/database/sql/sql.go:1067 +0xa7 fp=0xc000087fc0 sp=0xc000087f28 pc=0x11186e7
   database/sql.(*DB).startCleanerLocked.func1()
   	/usr/local/go/src/database/sql/sql.go:1054 +0x2a fp=0xc000087fe0 sp=0xc000087fc0 pc=0x111860a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000087fe8 sp=0xc000087fe0 pc=0xb18c21
   
   goroutine 56 [chan receive, 227 minutes]:
   runtime.gopark(0xc000608718?, 0xb1eda5?, 0x18?, 0x87?, 0xbe8066?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0006086f0 sp=0xc0006086d0 pc=0xae9996
   runtime.chanrecv(0xc000044d80, 0x0, 0x1)
   	/usr/local/go/src/runtime/chan.go:583 +0x49b fp=0xc000608780 sp=0xc0006086f0 pc=0xab54fb
   runtime.chanrecv1(0xc001960000?, 0x0?)
   	/usr/local/go/src/runtime/chan.go:442 +0x18 fp=0xc0006087a8 sp=0xc000608780 pc=0xab4ff8
   database/sql.(*Tx).awaitDone(0xc00191d580)
   	/usr/local/go/src/database/sql/sql.go:2187 +0x31 fp=0xc0006087c8 sp=0xc0006087a8 pc=0x111e991
   database/sql.(*DB).beginDC.func2()
   	/usr/local/go/src/database/sql/sql.go:1901 +0x26 fp=0xc0006087e0 sp=0xc0006087c8 pc=0x111d9e6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0006087e8 sp=0xc0006087e0 pc=0xb18c21
   
   goroutine 6 [sleep]:
   runtime.gopark(0x2b7b03a6e0671d?, 0x16c8625?, 0xe0?, 0xaa?, 0xc000052b00?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0019b7da0 sp=0xc0019b7d80 pc=0xae9996
   time.Sleep(0x3b9aca00)
   	/usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc0019b7de0 sp=0xc0019b7da0 pc=0xb15a15
   github.com/apache/incubator-devlake/server/services.RunPipelineInQueue(0x1)
   	/app/server/services/pipeline.go:211 +0x7cf fp=0xc0019b7fc8 sp=0xc0019b7de0 pc=0x1b8142f
   github.com/apache/incubator-devlake/server/services.pipelineServiceInit.func1()
   	/app/server/services/pipeline.go:116 +0x26 fp=0xc0019b7fe0 sp=0xc0019b7fc8 pc=0x1b807e6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0019b7fe8 sp=0xc0019b7fe0 pc=0xb18c21
   
   goroutine 2218 [select, 48 minutes]:
   runtime.gopark(0xc0019bff80?, 0x2?, 0xb8?, 0xff?, 0xc0019bff40?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0019b5dc0 sp=0xc0019b5da0 pc=0xae9996
   runtime.selectgo(0xc0019b5f80, 0xc0019bff3c, 0x1d41a80?, 0x0, 0xb15305?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc0019b5f00 sp=0xc0019b5dc0 pc=0xaf935c
   github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1()
   	/go/pkg/mod/github.com/go-sql-driver/mysql@v1.6.0/connection.go:614 +0xaa fp=0xc0019b5fe0 sp=0xc0019b5f00 pc=0x16e330a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0019b5fe8 sp=0xc0019b5fe0 pc=0xb18c21
   
   goroutine 2803 [IO wait]:
   runtime.gopark(0x8?, 0xb?, 0x0?, 0x0?, 0xb?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0019d7de8 sp=0xc0019d7dc8 pc=0xae9996
   runtime.netpollblock(0xb2f3c5?, 0xb90e45?, 0x0?)
   	/usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc0019d7e20 sp=0xc0019d7de8 pc=0xae1d77
   internal/poll.runtime_pollWait(0x7f5e3f78f828, 0x72)
   	/usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc0019d7e40 sp=0xc0019d7e20 pc=0xb12909
   internal/poll.(*pollDesc).wait(0xc0019ecc00?, 0xc00198cbe1?, 0x0)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0019d7e68 sp=0xc0019d7e40 pc=0xb91372
   internal/poll.(*pollDesc).waitRead(...)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Read(0xc0019ecc00, {0xc00198cbe1, 0x1, 0x1})
   	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a fp=0xc0019d7ee8 sp=0xc0019d7e68 pc=0xb9285a
   net.(*netFD).Read(0xc0019ecc00, {0xc00198cbe1?, 0xc001afe318?, 0xc0019d7f68?})
   	/usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc0019d7f30 sp=0xc0019d7ee8 pc=0xe1f7e9
   net.(*conn).Read(0xc0000143b8, {0xc00198cbe1?, 0xc0019560a0?, 0x1f0b605?})
   	/usr/local/go/src/net/net.go:183 +0x45 fp=0xc0019d7f78 sp=0xc0019d7f30 pc=0xe347e5
   net/http.(*connReader).backgroundRead(0xc00198cbd0)
   	/usr/local/go/src/net/http/server.go:678 +0x3f fp=0xc0019d7fc8 sp=0xc0019d7f78 pc=0xf383ff
   net/http.(*connReader).startBackgroundRead.func2()
   	/usr/local/go/src/net/http/server.go:674 +0x26 fp=0xc0019d7fe0 sp=0xc0019d7fc8 pc=0xf38326
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0019d7fe8 sp=0xc0019d7fe0 pc=0xb18c21
   ```
   
   ### Version
   
   v0.17.0-beta12
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] keon94 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1615091426

   PR to upgrade Go: https://github.com/apache/incubator-devlake/pull/5611
   Might be beneficial to upgrade the mapstructure to latest v1.5.0 (cc @klesh)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] keon94 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1615064202

   @awickham10 Not sure what to make of the stacktrace as the error is happening at a very low-level, with the Go runtime managing memory. If you can, perhaps try deploying v0.16 to see if that has this problem too? Or maybe wait for the first v0.18 beta which should be soon. I believe @klesh wants to upgrade Go to v1.20 (from 1.19), and that may help resolve this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] awickham10 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "awickham10 (via GitHub)" <gi...@apache.org>.
awickham10 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1638704265

   The root cause of this issue was the Dynatrace agent being injected into the container. Our monitoring team excluded the namespace from the Dynatrace deployments and this panic left the building. Thanks for your support with trying to upgrade Go and mapstructure!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] keon94 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1615110553

   Possibly related/similar: https://bugzilla.redhat.com/show_bug.cgi?id=1664386


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] keon94 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1614983519

   @awickham10 Is this just happening for webhook connections? Are the other ones working fine? Also, by frequently, do you mean it can still ocassionally work without this error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] awickham10 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "awickham10 (via GitHub)" <gi...@apache.org>.
awickham10 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1614986537

   > @awickham10 Is this just happening for webhook connections? Are the other ones working fine? Also, by frequently, do you mean it can still ocassionally work without this error?
   
   Same with Jenkins. I haven't been able to get a POST to work. Most of the time the GET will work, but sometimes it fails as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] awickham10 commented on issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "awickham10 (via GitHub)" <gi...@apache.org>.
awickham10 commented on issue #5612:
URL: https://github.com/apache/incubator-devlake/issues/5612#issuecomment-1615069586

   > @awickham10 Not sure what to make of the stacktrace as the error is happening at a very low-level, with the Go runtime managing memory. If you can, perhaps try deploying v0.16 to see if that has this problem too? Or maybe wait for the first v0.18 beta which should be soon. I believe @klesh wants to upgrade Go to v1.20 (from 1.19), and that may help resolve this.
   
   Looks like v0.16 was extremely similar.
   
   ```
   [GIN-debug] Listening and serving HTTP on 0.0.0.0:8080
   runtime: g 58: unexpected return pc for github.com/gin-gonic/gin.(*Engine).handleHTTPRequest called from 0x1cc50
   stack: frame={sp:0xc003009900, fp:0xc003009a88} stack=[0xc003006000,0xc00300a000)
   0x000000c003009800:  0x000000c0000818f0  0x000000046e54b479 
   0x000000c003009810:  0xc11fe6e20bea1508  0x0000000000000000 
   0x000000c003009820:  0x0000000000000000  0x000000c002954005 
   0x000000c003009830:  0x000000c000014018  0x0000000000000000 
   0x000000c003009840:  0x0000000001c6abc0  0x0000000000000000 
   0x000000c003009850:  0x00000000039bb6a0  0x0000000000000030 
   0x000000c003009860:  0x0000000000000013  0x0000000000000000 
   0x000000c003009870:  0x000000000000000b  0x000000c002954016 
   0x000000c003009880:  0x000000c0005a0731  0x0000000000a1daf2 <runtime.makeslice+0x0000000000000052> 
   0x000000c003009890:  0x000000000384e530  0x0000000000000080 
   0x000000c0030098a0:  0x0004110000203000  0x000000c00036f110 
   0x000000c0030098b0:  0x000000c002954016  0x000000c002878cf0 
   0x000000c0030098c0:  0x0000000000000005  0x0000000000000005 
   0x000000c0030098d0:  0x0000000000000000  0x0000000000000000 
   0x000000c0030098e0:  0x000000c0005a0720  0x000000000000001c 
   0x000000c0030098f0:  0x000000c003009a78  0x000000000133d6b1 <github.com/gin-gonic/gin.(*Engine).handleHTTPRequest+0x0000000000000671> 
   0x000000c003009900: <0x000000c002952100  0x000000c002954005 
   0x000000c003009910:  0x0000000000203000  0x000000c000012c78 
   0x000000c003009920:  0x000000c000012ca8  0x000000c002952100 
   0x000000c003009930:  0x00000000000003a0  0x0000000000000005 
   0x000000c003009940:  0x000000000000001c  0x0000000000000000 
   0x000000c003009950:  0x0000000000000004  0x0000000000000000 
   0x000000c003009960:  0x0000000000000000  0x000000c0000ee000 
   0x000000c003009970:  0x000000c0001f0460  0x000000c002954005 
   0x000000c003009980:  0x000000c002954000  0x000000c000101800 
   0x000000c003009990:  0x000000c000081a28  0x00007fd7442dcff8 
   0x000000c0030099a0:  0x00000000000000f8  0x00007fd7545725b8 
   0x000000c0030099b0:  0x0000000000000100  0x000000c0002a0000 
   0x000000c0030099c0:  0x000000c002952100  0x0000000000000000 
   0x000000c0030099d0:  0x000000c002878cf0  0x0000000000000005 
   0x000000c0030099e0:  0x0000000000000005  0x0000000000000000 
   0x000000c0030099f0:  0x000000000001ce00  0x000000c0005a0720 
   0x000000c003009a00:  0x000000000000001c  0x0000000001339789 <github.com/gin-gonic/gin.New.func1+0x00000000000000e9> 
   0x000000c003009a10:  0x0000000000a459eb <sync.(*Pool).getSlow+0x000000000000008b>  0x00000000039ecb70 
   0x000000c003009a20:  0x000000c000081a10  0x0000000000a38416 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c003009a30:  0x000000000001caf1  0x00007fd75725bcc0 
   0x000000c003009a40:  0x000000c002878cf0  0x0000000000000005 
   0x000000c003009a50:  0x0000000000000005  0x0000000000000000 
   0x000000c003009a60:  0x000000000001ce00  0x000000c0005a0720 
   0x000000c003009a70:  0x000000000000001c  0x000000c002b7dab8 
   0x000000c003009a80: !0x000000000001cc50 >0x000000c0001029c0 
   0x000000c003009a90:  0x000000c002952100  0x0000000000000000 
   0x000000c003009aa0:  0x000000c002952100  0x000000c000102ad8 
   0x000000c003009ab0:  0x0000000001e091a0  0x000000c002b7db68 
   0x000000c003009ac0:  0x0000000000e6264c <net/http.serverHandler.ServeHTTP+0x000000000000030c>  0x000000c0001029c0 
   0x000000c003009ad0:  0x0000000001e091a0  0x000000c002b74a80 
   0x000000c003009ae0:  0x000000c002952000  0x0000000000a4590e <sync.(*Pool).Get+0x000000000000008e> 
   0x000000c003009af0:  0x000100c000081b18  0x00000000009dc3a7 <runtime.newobject+0x0000000000000027> 
   0x000000c003009b00:  0x00007fd744348820  0x0000000001df9fc0 
   0x000000c003009b10:  0x000000c002948d2c  0x0000000000a38416 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c003009b20:  0x000000000001a511  0x000000c0001029c0 
   0x000000c003009b30:  0x000000c000081b40  0x000000c0026791d0 
   0x000000c003009b40:  0x000000000001a930  0x000000c000102b60 
   0x000000c003009b50:  0x000000000001a8d7  0x0000000000a383e0 <runtime.asmcgocall+0x0000000000000000> 
   0x000000c003009b60:  0x0000000000000000  0x000000c002b7dfa8 
   0x000000c003009b70:  0x000000000001a670  0x0000000001e00980 
   0x000000c003009b80:  0x0000000001e091a0 
   fatal error: unknown caller pc
   
   runtime stack:
   runtime.throw({0x7fd6bc176de3?, 0x2758a20?})
   	/usr/local/go/src/runtime/panic.go:1047 +0x5f fp=0x7fd7327fb758 sp=0x7fd7327fb728 pc=0x7fd6bba77e7f
   runtime.gentraceback(0x0?, 0x0?, 0x0?, 0x0?, 0x0, 0x0, 0x7fffffff, 0x1c6d948, 0x0?, 0x0)
   	/usr/local/go/src/runtime/traceback.go:269 +0x1c8a fp=0x7fd7327fbac0 sp=0x7fd7327fb758 pc=0x7fd6bba9a4ea
   runtime.copystack(0xc000102b60, 0x800000002?)
   	/usr/local/go/src/runtime/stack.go:932 +0x30d fp=0x7fd7327fbc78 sp=0x7fd7327fbac0 pc=0x7fd6bba8f9ad
   runtime.newstack()
   	/usr/local/go/src/runtime/stack.go:1112 +0x498 fp=0x7fd7327fbe30 sp=0x7fd7327fbc78 pc=0x7fd6bba8ff38
   runtime.morestack()
   	/usr/local/go/src/runtime/asm_amd64.s:570 +0x80 fp=0x7fd7327fbe38 sp=0x7fd7327fbe30 pc=0x7fd6bbaa1f80
   
   goroutine 58 [copystack]:
   runtime.heapBitsSetType(0xc000012cf0?, 0x18?, 0x18?, 0x1b68ae0?)
   	/usr/local/go/src/runtime/mbitmap.go:844 +0xc16 fp=0xc003008390 sp=0xc003008388 pc=0x7fd6bba573f6
   runtime.mallocgc(0x18, 0x1b68ae0, 0x1)
   	/usr/local/go/src/runtime/malloc.go:1050 +0x68e fp=0xc003008408 sp=0xc003008390 pc=0x7fd6bba4e30e
   runtime.makeslice(0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/slice.go:103 +0x52 fp=0xc003008430 sp=0xc003008408 pc=0x7fd6bba8d912
   reflect.Value.MapKeys({0x19b4300?, 0xc00294b4a0?, 0x0?})
   	/usr/local/go/src/reflect/value.go:1702 +0x105 fp=0xc003008508 sp=0xc003008430 pc=0x7fd6bbafada5
   github.com/mitchellh/mapstructure.(*Decoder).decodeStructFromMap(0xc00293e0a8, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?, 0x7fd75438bf00?}, {0x7fd6bc38a420?, 0xc00294c230?, 0x7fd75438bf00?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1243 +0x27d fp=0xc003008d50 sp=0xc003008508 pc=0x7fd6bbf8ae5d
   github.com/mitchellh/mapstructure.(*Decoder).decodeStruct(0x0?, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?}, {0x7fd6bc38a420?, 0xc00294c230?, 0x300?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1203 +0x5e8 fp=0xc003008e88 sp=0xc003008d50 pc=0x7fd6bbf8ab28
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc00293e0a8, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?}, {0x7fd6bc38a420?, 0xc00294c230?, 0x7fd6bbaa08d9?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:454 +0x329 fp=0xc003008f58 sp=0xc003008e88 pc=0x7fd6bbf819c9
   github.com/mitchellh/mapstructure.(*Decoder).decodePtr(0xc00293e0a8, {0x0, 0x0}, {0x19b4300, 0xc00294b4a0}, {0x7fd6bc340f60?, 0xc00293e0c0?, 0x7fd6bba4e35f?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1020 +0x44a fp=0xc003008ff0 sp=0xc003008f58 pc=0x7fd6bbf883ca
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc00293e0a8, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?}, {0x7fd6bc340f60?, 0xc00293e0c0?, 0xc00293e0b8?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:458 +0x2e9 fp=0xc0030090c0 sp=0xc003008ff0 pc=0x7fd6bbf81989
   github.com/mitchellh/mapstructure.(*Decoder).decodePtr(0xc00293e0a8, {0x0, 0x0}, {0x19b4300, 0xc00294b4a0}, {0xc002948e80?, 0xc00293e0c0?, 0x7fd7545725b8?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:1026 +0x4ea fp=0xc003009158 sp=0xc0030090c0 pc=0x7fd6bbf8846a
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc00293e0a8, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?}, {0xc002948e80?, 0xc00293e0c0?, 0xc002b71a30?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:458 +0x2e9 fp=0xc003009228 sp=0xc003009158 pc=0x7fd6bbf81989
   github.com/mitchellh/mapstructure.(*Decoder).decodeBasic(0x1988aa0?, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0}, {0x1988aa0?, 0xc002b71a40?, 0xc002948e00?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:504 +0x66c fp=0xc003009380 sp=0xc003009228 pc=0x7fd6bbf8252c
   github.com/mitchellh/mapstructure.(*Decoder).decode(0xc00293e0a8, {0x0, 0x0}, {0x19b4300?, 0xc00294b4a0?}, {0x1988aa0?, 0xc002b71a40?, 0x40?})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:444 +0x2c5 fp=0xc003009450 sp=0xc003009380 pc=0x7fd6bbf81965
   github.com/mitchellh/mapstructure.(*Decoder).Decode(0xc00293e0a8, {0x19b4300, 0xc00294b4a0})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:389 +0xea fp=0xc0030094b0 sp=0xc003009450 pc=0x7fd6bbf8166a
   github.com/mitchellh/mapstructure.Decode({0x19b4300, 0xc00294b4a0}, {0x1904240?, 0xc002b71a40})
   	/go/pkg/mod/github.com/mitchellh/mapstructure@v1.4.1/mapstructure.go:297 +0x95 fp=0xc0030094e0 sp=0xc0030094b0 pc=0x7fd6bbf81295
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.Decode({0x19b4300, 0xc00294b4a0}, {0x7fd6bc340f60, 0xc00294c230}, 0xc0002aec60)
   	/app/helpers/pluginhelper/api/mapstructure.go:85 +0xc6 fp=0xc003009538 sp=0xc0030094e0 pc=0x7fd6bc149a86
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*ConnectionApiHelper).merge(0xc000494e40, {0x7fd6bc340f60?, 0xc00294c230?}, 0x7fd6bba4e6c7?)
   	/app/helpers/pluginhelper/api/connection_helper.go:117 +0xe5 fp=0xc003009598 sp=0xc003009538 pc=0x7fd6bc148685
   github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*ConnectionApiHelper).Create(0xc000494e40, {0x7fd6bc340f60, 0xc00294c230}, 0xc0000a46d0?)
   	/app/helpers/pluginhelper/api/connection_helper.go:59 +0x30 fp=0xc0030095e0 sp=0xc003009598 pc=0x7fd6bc147f90
   github.com/apache/incubator-devlake/plugins/webhook/api.PostConnections(0xc00294b3e0?)
   	/app/plugins/webhook/api/connection.go:41 +0x52 fp=0xc003009620 sp=0xc0030095e0 pc=0x7fd6bc14e7f2
   github.com/apache/incubator-devlake/server/api.handlePluginCall.func1(0xc002952100)
   	/app/server/api/router.go:121 +0x1fe fp=0xc0030096f8 sp=0xc003009620 pc=0x18721be
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc002952100)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/recovery.go:99 +0x82 fp=0xc003009748 sp=0xc0030096f8 pc=0x133f682
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc002952100)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/logger.go:241 +0xe7 fp=0xc003009900 sp=0xc003009748 pc=0x133e7c7
   runtime: g 58: unexpected return pc for github.com/gin-gonic/gin.(*Engine).handleHTTPRequest called from 0x1cc50
   stack: frame={sp:0xc003009900, fp:0xc003009a88} stack=[0xc003006000,0xc00300a000)
   0x000000c003009800:  0x000000c0000818f0  0x000000046e54b479 
   0x000000c003009810:  0xc11fe6e20bea1508  0x0000000000000000 
   0x000000c003009820:  0x0000000000000000  0x000000c002954005 
   0x000000c003009830:  0x000000c000014018  0x0000000000000000 
   0x000000c003009840:  0x0000000001c6abc0  0x0000000000000000 
   0x000000c003009850:  0x00000000039bb6a0  0x0000000000000030 
   0x000000c003009860:  0x0000000000000013  0x0000000000000000 
   0x000000c003009870:  0x000000000000000b  0x000000c002954016 
   0x000000c003009880:  0x000000c0005a0731  0x0000000000a1daf2 <runtime.makeslice+0x0000000000000052> 
   0x000000c003009890:  0x000000000384e530  0x0000000000000080 
   0x000000c0030098a0:  0x0004110000203000  0x000000c00036f110 
   0x000000c0030098b0:  0x000000c002954016  0x000000c002878cf0 
   0x000000c0030098c0:  0x0000000000000005  0x0000000000000005 
   0x000000c0030098d0:  0x0000000000000000  0x0000000000000000 
   0x000000c0030098e0:  0x000000c0005a0720  0x000000000000001c 
   0x000000c0030098f0:  0x000000c003009a78  0x000000000133d6b1 <github.com/gin-gonic/gin.(*Engine).handleHTTPRequest+0x0000000000000671> 
   0x000000c003009900: <0x000000c002952100  0x000000c002954005 
   0x000000c003009910:  0x0000000000203000  0x000000c000012c78 
   0x000000c003009920:  0x000000c000012ca8  0x000000c002952100 
   0x000000c003009930:  0x00000000000003a0  0x0000000000000005 
   0x000000c003009940:  0x000000000000001c  0x0000000000000000 
   0x000000c003009950:  0x0000000000000004  0x0000000000000000 
   0x000000c003009960:  0x0000000000000000  0x000000c0000ee000 
   0x000000c003009970:  0x000000c0001f0460  0x000000c002954005 
   0x000000c003009980:  0x000000c002954000  0x000000c000101800 
   0x000000c003009990:  0x000000c000081a28  0x00007fd7442dcff8 
   0x000000c0030099a0:  0x00000000000000f8  0x00007fd7545725b8 
   0x000000c0030099b0:  0x0000000000000100  0x000000c0002a0000 
   0x000000c0030099c0:  0x000000c002952100  0x0000000000000000 
   0x000000c0030099d0:  0x000000c002878cf0  0x0000000000000005 
   0x000000c0030099e0:  0x0000000000000005  0x0000000000000000 
   0x000000c0030099f0:  0x000000000001ce00  0x000000c0005a0720 
   0x000000c003009a00:  0x000000000000001c  0x0000000001339789 <github.com/gin-gonic/gin.New.func1+0x00000000000000e9> 
   0x000000c003009a10:  0x0000000000a459eb <sync.(*Pool).getSlow+0x000000000000008b>  0x00000000039ecb70 
   0x000000c003009a20:  0x000000c000081a10  0x0000000000a38416 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c003009a30:  0x000000000001caf1  0x00007fd75725bcc0 
   0x000000c003009a40:  0x000000c002878cf0  0x0000000000000005 
   0x000000c003009a50:  0x0000000000000005  0x0000000000000000 
   0x000000c003009a60:  0x000000000001ce00  0x000000c0005a0720 
   0x000000c003009a70:  0x000000000000001c  0x000000c002b7dab8 
   0x000000c003009a80: !0x000000000001cc50 >0x000000c0001029c0 
   0x000000c003009a90:  0x000000c002952100  0x0000000000000000 
   0x000000c003009aa0:  0x000000c002952100  0x000000c000102ad8 
   0x000000c003009ab0:  0x0000000001e091a0  0x000000c002b7db68 
   0x000000c003009ac0:  0x0000000000e6264c <net/http.serverHandler.ServeHTTP+0x000000000000030c>  0x000000c0001029c0 
   0x000000c003009ad0:  0x0000000001e091a0  0x000000c002b74a80 
   0x000000c003009ae0:  0x000000c002952000  0x0000000000a4590e <sync.(*Pool).Get+0x000000000000008e> 
   0x000000c003009af0:  0x000100c000081b18  0x00000000009dc3a7 <runtime.newobject+0x0000000000000027> 
   0x000000c003009b00:  0x00007fd744348820  0x0000000001df9fc0 
   0x000000c003009b10:  0x000000c002948d2c  0x0000000000a38416 <runtime.asmcgocall+0x0000000000000036> 
   0x000000c003009b20:  0x000000000001a511  0x000000c0001029c0 
   0x000000c003009b30:  0x000000c000081b40  0x000000c0026791d0 
   0x000000c003009b40:  0x000000000001a930  0x000000c000102b60 
   0x000000c003009b50:  0x000000000001a8d7  0x0000000000a383e0 <runtime.asmcgocall+0x0000000000000000> 
   0x000000c003009b60:  0x0000000000000000  0x000000c002b7dfa8 
   0x000000c003009b70:  0x000000000001a670  0x0000000001e00980 
   0x000000c003009b80:  0x0000000001e091a0 
   github.com/gin-gonic/gin.(*Context).Next(...)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/context.go:168
   github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0001029c0, 0xc002952100)
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/gin.go:555 +0x671 fp=0xc003009a88 sp=0xc003009900 pc=0x133d6b1
   
   goroutine 1 [IO wait]:
   runtime.gopark(0x200000003?, 0xc0000061a0?, 0x0?, 0x65?, 0xc0021179c8?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc002117958 sp=0xc002117938 pc=0xa098b6
   runtime.netpollblock(0xc0021179a8?, 0x4?, 0x0?)
   	/usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc002117990 sp=0xc002117958 pc=0xa01d57
   internal/poll.runtime_pollWait(0x7fd7540f8ee8, 0x72)
   	/usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc0021179b0 sp=0xc002117990 pc=0xa32469
   internal/poll.(*pollDesc).wait(0xc002f7cc00?, 0x6?, 0x0)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0021179d8 sp=0xc0021179b0 pc=0xab07f2
   internal/poll.(*pollDesc).waitRead(...)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Accept(0xc002f7cc00)
   	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x234 fp=0xc002117a70 sp=0xc0021179d8 pc=0xab5bf4
   net.(*netFD).accept(0xc002f7cc00)
   	/usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc002117b28 sp=0xc002117a70 pc=0xd40495
   net.(*TCPListener).accept(0xc0002b3b30)
   	/usr/local/go/src/net/tcpsock_posix.go:142 +0x28 fp=0xc002117b58 sp=0xc002117b28 pc=0xd5cf28
   net.(*TCPListener).Accept(0xc0002b3b30)
   	/usr/local/go/src/net/tcpsock.go:288 +0x3d fp=0xc002117b88 sp=0xc002117b58 pc=0xd5bf9d
   net/http.(*onceCloseListener).Accept(0xc000236320?)
   	<autogenerated>:1 +0x2a fp=0xc002117ba0 sp=0xc002117b88 pc=0xe8b8ea
   net/http.(*Server).Serve(0xc0026791d0, {0x1e08f90, 0xc0002b3b30})
   	/usr/local/go/src/net/http/server.go:3070 +0x385 fp=0xc002117cd0 sp=0xc002117ba0 pc=0xe62c85
   net/http.(*Server).ListenAndServe(0xc0026791d0)
   	/usr/local/go/src/net/http/server.go:2999 +0x7d fp=0xc002117d00 sp=0xc002117cd0 pc=0xe627dd
   net/http.ListenAndServe(...)
   	/usr/local/go/src/net/http/server.go:3255
   github.com/gin-gonic/gin.(*Engine).Run(0xc0001029c0, {0xc002117e68, 0x1, 0x1})
   	/go/pkg/mod/github.com/gin-gonic/gin@v1.7.7/gin.go:356 +0x1d8 fp=0xc002117d90 sp=0xc002117d00 pc=0x133b758
   github.com/apache/incubator-devlake/server/api.CreateApiService()
   	/app/server/api/api.go:117 +0x625 fp=0xc002117f40 sp=0xc002117d90 pc=0x1871205
   main.main()
   	/app/server/main.go:39 +0x7e fp=0xc002117f80 sp=0xc002117f40 pc=0x187251e
   runtime.main()
   	/usr/local/go/src/runtime/proc.go:250 +0x212 fp=0xc002117fe0 sp=0xc002117f80 pc=0xa094f2
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc002117fe8 sp=0xc002117fe0 pc=0xa38781
   
   goroutine 2 [force gc (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000074fb0 sp=0xc000074f90 pc=0xa098b6
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.forcegchelper()
   	/usr/local/go/src/runtime/proc.go:302 +0xad fp=0xc000074fe0 sp=0xc000074fb0 pc=0xa0974d
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0xa38781
   
   goroutine 3 [GC sweep wait]:
   runtime.gopark(0x39bae01?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000075790 sp=0xc000075770 pc=0xa098b6
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.bgsweep(0x0?)
   	/usr/local/go/src/runtime/mgcsweep.go:297 +0xd7 fp=0xc0000757c8 sp=0xc000075790 pc=0x9f4457
   runtime.gcenable.func1()
   	/usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000757e0 sp=0xc0000757c8 pc=0x9e8fe6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000757e8 sp=0xc0000757e0 pc=0xa38781
   
   goroutine 4 [GC scavenge wait]:
   runtime.gopark(0xc0000a6000?, 0x1df2e00?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000075f70 sp=0xc000075f50 pc=0xa098b6
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.(*scavengerState).park(0x39bb720)
   	/usr/local/go/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc000075fa0 sp=0xc000075f70 pc=0x9f2433
   runtime.bgscavenge(0x0?)
   	/usr/local/go/src/runtime/mgcscavenge.go:622 +0x65 fp=0xc000075fc8 sp=0xc000075fa0 pc=0x9f2a45
   runtime.gcenable.func2()
   	/usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000075fe0 sp=0xc000075fc8 pc=0x9e8f86
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0xa38781
   
   goroutine 5 [finalizer wait]:
   runtime.gopark(0x39bc7e0?, 0xc000007860?, 0x0?, 0x0?, 0xc000074770?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000074628 sp=0xc000074608 pc=0xa098b6
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.runfinq()
   	/usr/local/go/src/runtime/mfinal.go:180 +0x10f fp=0xc0000747e0 sp=0xc000074628 pc=0x9e80ef
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000747e8 sp=0xc0000747e0 pc=0xa38781
   
   goroutine 6 [semacquire]:
   runtime.gopark(0xc0000766b8?, 0xa1a415?, 0x60?, 0x83?, 0xc0001c5c70?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000076668 sp=0xc000076648 pc=0xa098b6
   runtime.goparkunlock(...)
   	/usr/local/go/src/runtime/proc.go:369
   runtime.semacquire1(0xc000227458, 0x0?, 0x1, 0x0)
   	/usr/local/go/src/runtime/sema.go:150 +0x1fe fp=0xc0000766d0 sp=0xc000076668 pc=0xa1a15e
   sync.runtime_Semacquire(0xf42740?)
   	/usr/local/go/src/runtime/sema.go:62 +0x25 fp=0xc000076700 sp=0xc0000766d0 pc=0xa34a45
   sync.(*WaitGroup).Wait(0xc00024e230?)
   	/usr/local/go/src/sync/waitgroup.go:139 +0x52 fp=0xc000076728 sp=0xc000076700 pc=0xa46dd2
   github.com/spf13/viper.(*Viper).WatchConfig.func1()
   	/go/pkg/mod/github.com/spf13/viper@v1.8.1/viper.go:410 +0x305 fp=0xc0000767e0 sp=0xc000076728 pc=0xfdc265
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000767e8 sp=0xc0000767e0 pc=0xa38781
   
   goroutine 7 [syscall]:
   syscall.Syscall6(0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/syscall/syscall_linux.go:90 +0x36 fp=0xc00028fb18 sp=0xc00028fa90 pc=0xa50dd6
   syscall.Syscall6(0xe8, 0x10, 0xc00028fc14, 0x7, 0xffffffffffffffff, 0x0, 0x0)
   	<autogenerated>:1 +0x45 fp=0xc00028fb60 sp=0xc00028fb18 pc=0xa51185
   golang.org/x/sys/unix.EpollWait(0x0?, {0xc00028fc14?, 0x0?, 0x0?}, 0x0?)
   	/go/pkg/mod/golang.org/x/sys@v0.1.0/unix/zsyscall_linux_amd64.go:56 +0x58 fp=0xc00028fbd0 sp=0xc00028fb60 pc=0xeafd98
   github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc000230c00)
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify_poller.go:87 +0x7d fp=0xc00028fc80 sp=0xc00028fbd0 pc=0xf43a3d
   github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc00024e230)
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify.go:193 +0x275 fp=0xc00029ffc8 sp=0xc00028fc80 pc=0xf42d15
   github.com/fsnotify/fsnotify.NewWatcher.func1()
   	/go/pkg/mod/github.com/fsnotify/fsnotify@v1.5.1/inotify.go:60 +0x26 fp=0xc00029ffe0 sp=0xc00029ffc8 pc=0xf422e6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00029ffe8 sp=0xc00029ffe0 pc=0xa38781
   
   goroutine 8 [select]:
   runtime.gopark(0xc0000777b0?, 0x2?, 0x0?, 0x0?, 0xc000077734?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000080d98 sp=0xc000080d78 pc=0xa098b6
   runtime.selectgo(0xc000080fb0, 0xc000077730, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000080ed8 sp=0xc000080d98 pc=0xa18edc
   github.com/spf13/viper.(*Viper).WatchConfig.func1.1()
   	/go/pkg/mod/github.com/spf13/viper@v1.8.1/viper.go:371 +0x105 fp=0xc000080fe0 sp=0xc000080ed8 pc=0xfdc3c5
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000080fe8 sp=0xc000080fe0 pc=0xa38781
   
   goroutine 9 [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000077f50 sp=0xc000077f30 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000077fe0 sp=0xc000077f50 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000077fe8 sp=0xc000077fe0 pc=0xa38781
   
   goroutine 18 [GC worker (idle)]:
   runtime.gopark(0x0?, 0xc000047c60?, 0x0?, 0x0?, 0xc0001f2ee0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000077750 sp=0xc000077730 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000777e0 sp=0xc000077750 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000777e8 sp=0xc0000777e0 pc=0xa38781
   
   goroutine 10 [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000070750 sp=0xc000070730 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000707e0 sp=0xc000070750 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000707e8 sp=0xc0000707e0 pc=0xa38781
   
   goroutine 11 [GC worker (idle)]:
   runtime.gopark(0x2d6c0efe39b321?, 0x3?, 0x1a?, 0xac?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000070f50 sp=0xc000070f30 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000070fe0 sp=0xc000070f50 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000070fe8 sp=0xc000070fe0 pc=0xa38781
   
   goroutine 12 [GC worker (idle)]:
   runtime.gopark(0x2d6c0f0f45b8fe?, 0x1?, 0xf6?, 0x42?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000071750 sp=0xc000071730 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000717e0 sp=0xc000071750 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000717e8 sp=0xc0000717e0 pc=0xa38781
   
   goroutine 13 [GC worker (idle)]:
   runtime.gopark(0x2d6c0f0f452587?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0022dcf50 sp=0xc0022dcf30 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0022dcfe0 sp=0xc0022dcf50 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0022dcfe8 sp=0xc0022dcfe0 pc=0xa38781
   
   goroutine 14 [GC worker (idle)]:
   runtime.gopark(0x2d6c0efe397d4e?, 0x0?, 0x0?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000072750 sp=0xc000072730 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000727e0 sp=0xc000072750 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0xa38781
   
   goroutine 15 [GC worker (idle)]:
   runtime.gopark(0x2d6c0efe39630e?, 0x1?, 0xa1?, 0x2f?, 0x0?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000072f50 sp=0xc000072f30 pc=0xa098b6
   runtime.gcBgMarkWorker()
   	/usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000072fe0 sp=0xc000072f50 pc=0x9eb131
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0xa38781
   
   goroutine 16 [chan receive]:
   runtime.gopark(0xc0002ae120?, 0xa0f71b?, 0xf0?, 0xde?, 0xa259e5?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc00001de98 sp=0xc00001de78 pc=0xa098b6
   runtime.chanrecv(0xc0002ae0c0, 0xc00001df98, 0x1)
   	/usr/local/go/src/runtime/chan.go:583 +0x49b fp=0xc00001df28 sp=0xc00001de98 pc=0x9d54fb
   runtime.chanrecv2(0xc0001d845c?, 0x0?)
   	/usr/local/go/src/runtime/chan.go:447 +0x18 fp=0xc00001df50 sp=0xc00001df28 pc=0x9d5038
   github.com/panjf2000/ants/v2.(*Pool).purgePeriodically(0xc00031d180)
   	/go/pkg/mod/github.com/panjf2000/ants/v2@v2.4.6/pool.go:69 +0x8b fp=0xc00001dfc8 sp=0xc00001df50 pc=0x11f474b
   github.com/panjf2000/ants/v2.NewPool.func2()
   	/go/pkg/mod/github.com/panjf2000/ants/v2@v2.4.6/pool.go:137 +0x26 fp=0xc00001dfe0 sp=0xc00001dfc8 pc=0x11f4ce6
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00001dfe8 sp=0xc00001dfe0 pc=0xa38781
   
   goroutine 50 [select]:
   runtime.gopark(0xc000025f88?, 0x2?, 0x11?, 0x1?, 0xc000025f84?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000147e10 sp=0xc000147df0 pc=0xa098b6
   runtime.selectgo(0xc000147f88, 0xc000025f80, 0x19200e0?, 0x0, 0xc000025fd0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000147f50 sp=0xc000147e10 pc=0xa18edc
   database/sql.(*DB).connectionOpener(0xc0003cba00, {0x1e0a5d0, 0xc0005bfdc0})
   	/usr/local/go/src/database/sql/sql.go:1224 +0x8d fp=0xc000147fb8 sp=0xc000147f50 pc=0xff3ead
   database/sql.OpenDB.func1()
   	/usr/local/go/src/database/sql/sql.go:792 +0x2e fp=0xc000147fe0 sp=0xc000147fb8 pc=0xff216e
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000147fe8 sp=0xc000147fe0 pc=0xa38781
   
   goroutine 19 [select]:
   runtime.gopark(0xc000023f80?, 0x2?, 0x0?, 0x0?, 0xc000023f40?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000085dc0 sp=0xc000085da0 pc=0xa098b6
   runtime.selectgo(0xc000085f80, 0xc000023f3c, 0xc000023f70?, 0x0, 0xc0003cbad0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000085f00 sp=0xc000085dc0 pc=0xa18edc
   github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1()
   	/go/pkg/mod/github.com/go-sql-driver/mysql@v1.6.0/connection.go:614 +0xaa fp=0xc000085fe0 sp=0xc000085f00 pc=0x13cc40a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0xa38781
   
   goroutine 35 [select]:
   runtime.gopark(0xc000197f88?, 0x2?, 0x0?, 0x0?, 0xc000197f84?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000086e10 sp=0xc000086df0 pc=0xa098b6
   runtime.selectgo(0xc000086f88, 0xc000197f80, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000086f50 sp=0xc000086e10 pc=0xa18edc
   database/sql.(*DB).connectionOpener(0xc00025e0d0, {0x1e0a5d0, 0xc0000534c0})
   	/usr/local/go/src/database/sql/sql.go:1224 +0x8d fp=0xc000086fb8 sp=0xc000086f50 pc=0xff3ead
   database/sql.OpenDB.func1()
   	/usr/local/go/src/database/sql/sql.go:792 +0x2e fp=0xc000086fe0 sp=0xc000086fb8 pc=0xff216e
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000086fe8 sp=0xc000086fe0 pc=0xa38781
   
   goroutine 20 [select]:
   runtime.gopark(0xc000193f90?, 0x2?, 0x0?, 0x30?, 0xc000193f74?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000142de8 sp=0xc000142dc8 pc=0xa098b6
   runtime.selectgo(0xc000142f90, 0xc000193f70, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000142f28 sp=0xc000142de8 pc=0xa18edc
   database/sql.(*DB).connectionCleaner(0xc0003cba00, 0x0?)
   	/usr/local/go/src/database/sql/sql.go:1067 +0xa7 fp=0xc000142fc0 sp=0xc000142f28 pc=0xff3287
   database/sql.(*DB).startCleanerLocked.func1()
   	/usr/local/go/src/database/sql/sql.go:1054 +0x2a fp=0xc000142fe0 sp=0xc000142fc0 pc=0xff31aa
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000142fe8 sp=0xc000142fe0 pc=0xa38781
   
   goroutine 39 [select]:
   runtime.gopark(0xc000021f80?, 0x2?, 0xd0?, 0xa5?, 0xc000021f40?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000143dc0 sp=0xc000143da0 pc=0xa098b6
   runtime.selectgo(0xc000143f80, 0xc000021f3c, 0xc000021f70?, 0x0, 0xc0003cbad0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000143f00 sp=0xc000143dc0 pc=0xa18edc
   github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1()
   	/go/pkg/mod/github.com/go-sql-driver/mysql@v1.6.0/connection.go:614 +0xaa fp=0xc000143fe0 sp=0xc000143f00 pc=0x13cc40a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000143fe8 sp=0xc000143fe0 pc=0xa38781
   
   goroutine 40 [chan receive]:
   runtime.gopark(0xc00023ff18?, 0xa3e905?, 0x18?, 0xff?, 0xb07486?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000146ef0 sp=0xc000146ed0 pc=0xa098b6
   runtime.chanrecv(0xc000044c60, 0x0, 0x1)
   	/usr/local/go/src/runtime/chan.go:583 +0x49b fp=0xc000146f80 sp=0xc000146ef0 pc=0x9d54fb
   runtime.chanrecv1(0xc000053c40?, 0xc00024f3b0?)
   	/usr/local/go/src/runtime/chan.go:442 +0x18 fp=0xc000146fa8 sp=0xc000146f80 pc=0x9d4ff8
   database/sql.(*Tx).awaitDone(0xc00038be00)
   	/usr/local/go/src/database/sql/sql.go:2187 +0x31 fp=0xc000146fc8 sp=0xc000146fa8 pc=0xff9531
   database/sql.(*DB).beginDC.func2()
   	/usr/local/go/src/database/sql/sql.go:1901 +0x26 fp=0xc000146fe0 sp=0xc000146fc8 pc=0xff8586
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000146fe8 sp=0xc000146fe0 pc=0xa38781
   
   goroutine 54 [select]:
   runtime.gopark(0xc00001ef90?, 0x2?, 0x0?, 0x30?, 0xc00001ef74?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc00001ede8 sp=0xc00001edc8 pc=0xa098b6
   runtime.selectgo(0xc00001ef90, 0xc00001ef70, 0x0?, 0x0, 0x0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc00001ef28 sp=0xc00001ede8 pc=0xa18edc
   database/sql.(*DB).connectionCleaner(0xc00025e0d0, 0x1c?)
   	/usr/local/go/src/database/sql/sql.go:1067 +0xa7 fp=0xc00001efc0 sp=0xc00001ef28 pc=0xff3287
   database/sql.(*DB).startCleanerLocked.func1()
   	/usr/local/go/src/database/sql/sql.go:1054 +0x2a fp=0xc00001efe0 sp=0xc00001efc0 pc=0xff31aa
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00001efe8 sp=0xc00001efe0 pc=0xa38781
   
   goroutine 42 [select]:
   runtime.gopark(0xc000027f80?, 0x2?, 0xd0?, 0xa5?, 0xc000027f40?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000144dc0 sp=0xc000144da0 pc=0xa098b6
   runtime.selectgo(0xc000144f80, 0xc000027f3c, 0xc000027f70?, 0x0, 0xc0003cbad0?, 0x1)
   	/usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000144f00 sp=0xc000144dc0 pc=0xa18edc
   github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1()
   	/go/pkg/mod/github.com/go-sql-driver/mysql@v1.6.0/connection.go:614 +0xaa fp=0xc000144fe0 sp=0xc000144f00 pc=0x13cc40a
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000144fe8 sp=0xc000144fe0 pc=0xa38781
   
   goroutine 59 [IO wait]:
   runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x18?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000087de8 sp=0xc000087dc8 pc=0xa098b6
   runtime.netpollblock(0xa4ed65?, 0x0?, 0x0?)
   	/usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc000087e20 sp=0xc000087de8 pc=0xa01d57
   internal/poll.runtime_pollWait(0x7fd7540f8df8, 0x72)
   	/usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc000087e40 sp=0xc000087e20 pc=0xa32469
   internal/poll.(*pollDesc).wait(0xc00020f680?, 0xc00294b391?, 0x0)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000087e68 sp=0xc000087e40 pc=0xab07f2
   internal/poll.(*pollDesc).waitRead(...)
   	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Read(0xc00020f680, {0xc00294b391, 0x1, 0x1})
   	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a fp=0xc000087ee8 sp=0xc000087e68 pc=0xab1cda
   net.(*netFD).Read(0xc00020f680, {0xc00294b391?, 0x0?, 0x0?})
   	/usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc000087f30 sp=0xc000087ee8 pc=0xd3e329
   net.(*conn).Read(0xc00293e0a0, {0xc00294b391?, 0xb06bb0?, 0xc000495840?})
   	/usr/local/go/src/net/net.go:183 +0x45 fp=0xc000087f78 sp=0xc000087f30 pc=0xd532a5
   net/http.(*connReader).backgroundRead(0xc00294b380)
   	/usr/local/go/src/net/http/server.go:678 +0x3f fp=0xc000087fc8 sp=0xc000087f78 pc=0xe56e1f
   net/http.(*connReader).startBackgroundRead.func2()
   	/usr/local/go/src/net/http/server.go:674 +0x26 fp=0xc000087fe0 sp=0xc000087fc8 pc=0xe56d46
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000087fe8 sp=0xc000087fe0 pc=0xa38781
   
   goroutine 44 [sleep]:
   runtime.gopark(0x2d6c1357f9722f?, 0x13b17c5?, 0x80?, 0x97?, 0xc0022e9580?)
   	/usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc002b7bda0 sp=0xc002b7bd80 pc=0xa098b6
   time.Sleep(0x3b9aca00)
   	/usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc002b7bde0 sp=0xc002b7bda0 pc=0xa35575
   github.com/apache/incubator-devlake/server/services.RunPipelineInQueue(0x1)
   	/app/server/services/pipeline.go:203 +0x7cf fp=0xc002b7bfc8 sp=0xc002b7bde0 pc=0x185a1cf
   github.com/apache/incubator-devlake/server/services.pipelineServiceInit.func1()
   	/app/server/services/pipeline.go:108 +0x26 fp=0xc002b7bfe0 sp=0xc002b7bfc8 pc=0x1859586
   runtime.goexit()
   	/usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc002b7bfe8 sp=0xc002b7bfe0 pc=0xa38781
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] awickham10 closed issue #5612: [Bug] Go Panic in OpenShift Kubernetes

Posted by "awickham10 (via GitHub)" <gi...@apache.org>.
awickham10 closed issue #5612: [Bug] Go Panic in OpenShift Kubernetes
URL: https://github.com/apache/incubator-devlake/issues/5612


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org