More Effective Go

Unbounded Iteration

“Unbounded iteration” is when you need to iterate over a sequence without knowing its total length. For example, receiving rows from a database query or data chunks from an HTTP response. Other languages have a native concept of “iterators” such that iterating over an array and stream use the same syntax, but Go doesn’t do this.

The best approach to unbounded iteration in Go is a callback:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
func stream(cb func(int)) {
	for _, x := range []int{1, 2, 3} {
		cb(x)
		time.Sleep(time.Second)
	}
}

func main() {
	stream(func(x int) {
		fmt.Println(x)
	})
}

Some developers new to Go may try to use a channel and background thread for unbounded iteration. Don’t do this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
func stream() <-chan int {
	ch := make(chan int)
	go func() {
		defer close(ch)
		for _, x := range []int{1, 2, 3} {
			ch <- x
			time.Sleep(time.Second)
		}
	}()
	return ch
}

func main() {
	for x := range stream() {
		fmt.Println(x)
	}
}

Threads and thread-safe communication are cheap in Go, but not free. They add runtime and mental overhead – you need to think about the lifetime of any temporary channels backing your loops, and make sure they get properly drained so their backing threads can terminate.

The channel approach is also more difficult to extend. If a callback needs to change to return an error, or a non-error early exit, it’s straightforward to add a return type. Channels have no mechanism to return data from the receiver.

Option Interfaces

Keyword arguments (“kwargs”) are commonly used in other languages for passing optional parameters to a complicated API. Go doesn’t have kwargs, and they can be awkward to imitate using an “options struct” because the receiver can’t easily tell whether an option was explicitly set to its zero value:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
type Options struct {
	ConcurrencyToken uint32
}

func Fetch(opts Options) {
	if opts.ConcurrencyToken == 0 {
		// is this fetch being run without a concurrency token? or did the
		// caller set a token, but it happens to be 0x00000000 ?
	}
}

Option interfaces let the options themselves be defined by functions, so that presence/absence, validation, and complex defaults are expressed naturally (with a cost of increased boilerplate):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
type Option interface {
	apply(*options)
}

type fnOption func(*options)
func (fn fnOption) apply(opts *options) { fn(opts) }

type options struct {
	concurrencyToken *uint32
}

func ConcurrencyToken(token uint32) Option {
	return fnOption(func(opts *options) {
		opts.concurrencyToken = &token
	})
}

func Fetch(opts ...Option) {
	appliedOpts := options{}
	for _, opt := range opts {
		opt.apply(&appliedOpts)
	}
	if appliedOpts.ConcurrencyToken == nil {
		// definitely being run without a concurrency token
	}
}

Prefer POSIX Flags

Go’s standard library contains a flags package for parsing command-line flags. It uses Plan 9 flag semantics, which are alien to advanced users with a Linux, UNIX, or Windows background (i.e. all of them).

The github.com/spf13/pflag library is API-compatible with the stdlib flags package, has extra API for features like “short” flags, and can automatically import flag definitions from libraries that use the stdlib.

1
2
3
4
5
6
7
8
9
import (
	goflag "flag"
	flag "github.com/spf13/pflag"
)

func main() {
	flag.CommandLine.AddGoFlagSet(goflag.CommandLine)
	flag.Parse()
}

Dynamic Flag Defaults

Sometimes you’ll want a command-line flag with a default value that can’t be hardcoded, like --config-path that defaults to somewhere in the user’s home directory. A common patttern is to let the flag’s zero value mean “use computed default”, but this makes --help output less useful.

It’s better to use a computed value for the flag’s default at definition time, then (1) --help will show that default value and (2) code consuming the flag doesn’t need to special-case it.

1
2
3
4
5
6
7
8
9
configPath = flag.String("config-path", defaultConfigPath(), "[your wonderful documentation here]")

func defaultConfigPath() {
	path := os.ExpandEnv("${HOME}/.config/my-client-config"
	if _, err := os.Stat(path); err == nil {
		return path
	}
	return ""
}

Errors Should Include Stack Traces

If your code constructs errors with fmt.Errorf() or similar standard library functions, you’re implicitly dropping the stack trace of where that error happened. Prefer to use the github.com/pkg/errors library, which records stack traces when the error is created and can preserve them as explanatory text is added in callers.

Custom error types can also use this library to obtain and propagate stack traces:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
import "github.com/pkg/errors"

type myCustomError struct {
	code int32
	trace errors.StackTrace
}

func (err *myCustomError) StackTrace() errors.StackTrace {
	return err.trace
}

func fail(code int32) error {
	trace := errors.New("").(stackTrace).StackTrace()
	return &myCustomError{
		code: code,
		trace: trace[1:],
	}
}

Avoid Mutable Globals

This is standard good programming practice, but I want to specifically call it out here because the Go standard library is full of these things. You must be careful.

For example, the net/http package has functions Handle(), ListenAndServe(), etc that operate on http.DefaultServeMux. You don’t want to use these. Prefer to explicitly create your own *http.ServeMux and pass it around as an explicit parameter. Then when you want to write tests you won’t need to go back and figure out all the places you’re poking at mutable global state.

1
2
3
4
5
6
7
8
// BAD
http.Handle("/foo", fooHandler)
http.ListenAndServe(":8080", nil)

// GOOD
mux := http.NewServeMux()
mux.Handle("/foo", fooHandler)
http.ListenAndServe(":8080", mux)

Don’t Mutate or Invalidate Parameters

This is a hard rule for your public API. It’s also helpful to comply with in private APIs, but you don’t need to if you’re willing to accept the risk of weird bugs.

A public function defined like Listen(addrs []string) shouldn’t mutate or invalidate the value passed in for addrs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
// BAD!
func Listen(addrs []string) {
	for ii, addr := range addrs {
		addrs[ii] = addr + ":1234"
	}
}


// BAD!
func Listen(addrs []string) {
	addrs = append(addrs, "localhost:1234")
}

If you need to make adjustments to a user-provided value, copy it first:

1
2
3
4
5
6
func Listen(addrs []string) {
	addrs = append([]string{}, addrs...)

	// safe
	addrs = append(addrs, "localhost:1234")
}