Background
When starting the process, it occasionally crashes. If it doesn’t crash at startup, it won’t crash during subsequent operations. This narrows down the debugging scope. Below is part of the error stack information. It is clear that the crash is caused by concurrent map writes, as maps in Golang are not concurrency-safe.
1fatal error: concurrent map writes
2
3goroutine 11 [running]:
4github.com/zeromicro/go-zero/core/search.add(0xc00008ed20, {0xc000156d09, 0x16}, {0x1268c00, 0xc0003845a0})
5 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/core/search/tree.go:182 +0x2b5
6github.com/zeromicro/go-zero/core/search.(*Tree).Add(0x12581c0?, {0xc000156d08, 0x17}, {0x1268c00?, 0xc0003845a0?})
7 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/core/search/tree.go:71 +0x75
8github.com/zeromicro/go-zero/rest/router.(*patRouter).Handle(0xc00041d230, {0x136ccf8, 0x4}, {0xc000156d08?, 0xc0004121c0?}, {0x148c5a0, 0xc0003845a0})
9 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/rest/router/patrouter.go:51 +0x190
10github.com/zeromicro/go-zero/rest.(*engine).bindRoute(0xc000162608, {0x37e11d600, 0x0, {0x0, {0x0, 0x0}, {0x0, 0x0}}, {{0x0, 0x0, ...}, ...}, ...}, ...)
11 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/rest/engine.go:111 +0x29a
12github.com/zeromicro/go-zero/rest.(*engine).bindFeaturedRoutes(0xc000162608, {0x14943b0, 0xc00010c8a0}, {0x37e11d600, 0x0, {0x0, {0x0, 0x0}, {0x0, 0x0}}, ...}, ...)
13 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/rest/engine.go:89 +0x18d
14github.com/zeromicro/go-zero/rest.(*engine).bindRoutes(0xc000162608, {0x14943b0, 0xc00010c8a0})
15 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/rest/engine.go:118 +0x11a
16github.com/zeromicro/go-zero/rest.(*Server).ServeHTTP(0xc00010c870, {0x1492cd0, 0xc00000c1c0}, 0xc000130240)
17 C:/Users/hanleng/go/pkg/mod/github.com/zeromicro/[email protected]/rest/server.go:112 +0x3b
18net/http.serverHandler.ServeHTTP({0xc00058c4e0?}, {0x1492cd0?, 0xc00000c1c0?}, 0x6?)
19 C:/Users/hanleng/.g/go/src/net/http/server.go:3137 +0x8e
20net/http.(*conn).serve(0xc00019a750, {0x1494148, 0xc00048e2a0})
21 C:/Users/hanleng/.g/go/src/net/http/server.go:2039 +0x5e8
22created by net/http.(*Server).Serve in goroutine 50
23 C:/Users/hanleng/.g/go/src/net/http/server.go:3285 +0x4b4
24net/http.serverHandler.ServeHTTP({0xc00058c4e0?}, {0x1492cd0?, 0xc00000c1c0?}, 0x6?)
25 C:/Users/hanleng/.g/go/src/net/http/server.go:3137 +0x8e
26net/http.(*conn).serve(0xc00019a750, {0x1494148, 0xc00048e2a0})
27 C:/Users/hanleng/.g/go/src/net/http/server.go:2039 +0x5e8
28created by net/http.(*Server).Serve in goroutine 50
29 C:/Users/hanleng/.g/go/src/net/http/server.go:3285 +0x4b4
However, the key issue is that the code related to startup does not use a map. Where does the map causing the panic come from? Initially, because the error stack information was all from the Go-Zero framework’s source code, I didn’t think much about it and still wanted to find the cause within my own code. To locate the map, I added -race during execution to perform race detection, which allows seeing the time and location of the creation of the competing goroutines. Eventually, it led back to Go-Zero. According to the error stack information, the code line pointed to by tree.go:182 is as follows:
1children[token] = child // children type is map[string]*node
Tracing back,
1func main() {
2 // Start HTTP service
3 srv := &http.Server{
4 Addr: httpAddr,
5 Handler: server,
6 }
7 go func() {
8 srv.ListenAndServe()
9 }()
10 // Start WebSocket service
11 go func() {
12 http.ListenAndServe(wsAddr, nil)
13 }()
14}
When the program starts, it launches two goroutines to start the websocket service and the http service, both eventually calling the ListenAndServe method. There is a probability of concurrent operations on a certain map in the go-zero framework, which is not locked in the framework, causing the panic.