Skip to content

Go Language

When you have 2 keys for a value, 1d or 2d map should you use?

The topic was found during optimization. When 2 string type keys indicate a value, how do you maintain the mapping? Generally we can come up with 3 approaches listed below:

type Result string

// 1d map, you need to concatenate 2 strings
var d1M map[string]Result

// 1d map and use struct as key
type Key struct {
    S1, S2 string
}
var d1MS map[Key]Result

// 2d map
var d2M map[string]map[string]Result

The 1d mapping needs pay for the string concatenation, the 2d mapping is required for an additional map query per query and the 1d map with struct key requires additional calculation on map.

The conclusion is 2d map is fastest because string concatenation costs more. What's worse, inappropriate string concatenations causes higher memory usages and triggers more GC(result got from pprof). It has been verified in the real practice by my teammates as well.

Note that here the keys are string. If they're small integers we can consider to use bit operation to append them together. Hence, we can finish the map querying in 1 operation.

This blog is not intended to analyze which method to append string(fmt.Sprintf, +, strings.Builder, etc...) is better and why it's fastest. It shows the result only.

How Go Finds Std Libs

Usually, the go standard library is located under $GOROOT and it's go compiler's scope to find the std libs, so most of the users needn't care about how the standard library is found by compiler.

When we want to combine 2 services into a single process, we need to solve the collision of global states between 2 services in the standard library. By duplicating it with another name(and all underlying dependent packages) under GOROOT, we can segregate the global states well even though they refer to the same package before combining. However, this requires to investigate how go compiler finds the std lib to ensure the approach won't break.

golang/tools: reduce jitter of packages.Load

In CL 614975, I have migrated deprecated loader from ssa package. However, miller has reported the tests became extremely slows. In an operation-system with a slow file system, such as plan9, the test slowed by more than 50x.

Oops, a un-intentional change:( I have submitted a CL to load 300+ packages by packages.Load once instead of loading them many. Alan thought it's a bravo change:)

Moreover, when I involved to investigate why the packages.Load slows so much, I have learned a lot. This blog records how I investigate them.