Unit Test Using Commands Instead Of Mocks 🔗

Say you wrote some code to benchmark incremental builds:

func Benchmark(
    binary string, sourcePath string,
    resultPath string, numBuilds int) error {
  if err := build(binary); err != nil {
    return err
  }
  source, err := read(sourcePath)
  if err != nil {
    return err
  }
  deltas := []time.Duration{}
  for i := 0; i < numBuilds; i++ {
    if err := write(
        sourcePath,
        edit(source, i)); err != nil {
      return err
    }
    start := now()
    if err := build(binary); err != nil {
      return err
    }
    delta := now().Sub(start)
    deltas = append(deltas, delta)
    if err := revert(sourcePath); err != nil {
      return err
    }
  }
  return write(resultPath, report(deltas))
}

func Main() error {
  // these values would come from flags
  return Benchmark(
    "binary", "source_path",
    "result_path", 3 /* numBuilds */)
}

// compiler
func build(binary string) error {
  /* ... */
}

// file system
func read(path string) (string, error) {
  /* ... */
}

// self-contained
func edit(content string, delta int) string {
  /* ... */
}

// file system
func write(path string, content string) error {
  /* ... */
}

// time
func now() time.Time {
  return time.Now()
}

// version control
func revert(path string) error {
  /* ... */
}

// self-contained
func report(deltas []time.Duration) string {
  /* ... */
}

Explanation:

build a binary
read one of its source files
for each iteration
- edit the source in memory and write it back to disk
- measure how long the incremental build takes
- revert the source file using the version control
write all measurements

How would you test the Benchmark function? It has multiple external dependencies:

compiler (to build)
file system (to read and write files)
system clock (to measure the elapsed time)
version control (to revert the edited source file)

Some people would only unit test the functions edit and report, which don’t have external dependencies, and not the rest.

Move More Code To Main 🔗

Sometimes it is possible to move more code to the Main function to make the remaining code easier to test:

func Benchmark(
    binary string,
    sourcePath string,
    source string,
    numBuilds int) (string, error) {
  deltas := []time.Duration{}
  for i := 0; i < numBuilds; i++ {
    if err := write(
        sourcePath,
        edit(source, i)); err != nil {
      return "", err
    }
    start := now()
    if err := build(binary); err != nil {
      return "", err
    }
    delta := now().Sub(start)
    deltas = append(deltas, delta)
    if err := revert(sourcePath); err != nil {
      return "", err
    }
  }
  return report(deltas), nil
}

func Main() error {
  // these values would come from flags
  binary := "binary"
  sourcePath := "source_path"
  resultPath := "result_path"
  if err := build(binary); err != nil {
    return err
  }
  source, err := read(sourcePath)
  if err != nil {
    return err
  }
  result, err := Benchmark(
    binary, sourcePath,
    source, 3 /* numBuilds */)
  if err != nil {
    return err
  }
  return write(resultPath, result)
}

In this case doing this is not enough.

Use Dependency Injection 🔗

One way to make this code more testable (with fast, reliable, self-contained unit tests) is to use dependency injection and fakes. Normally you would inject a separate parameter for Compiler, FileSystem, Timer, VersionControl. I’ve injected a single parameter Delegate to simplify:

type Delegate interface {
  Build(binary string) error
  Read(path string) (string, error)
  Write(path string, content string) error
  Now() time.Time
  Revert(path string) error
}

type Benchmark struct {
  Delegate Delegate
}

func (b *Benchmark) Benchmark(
    binary string, sourcePath string,
    resultPath string, numBuilds int) error {
  if err := b.Delegate.Build(
      binary); err != nil {
    return err
  }
  source, err := b.Delegate.Read(sourcePath)
  if err != nil {
    return err
  }
  deltas := []time.Duration{}
  for i := 0; i < numBuilds; i++ {
    if err := b.Delegate.Write(
        sourcePath,
        edit(source, i)); err != nil {
      return err
    }
    start := b.Delegate.Now()
    if err := b.Delegate.Build(
        binary); err != nil {
      return err
    }
    delta := b.Delegate.Now().Sub(start)
    deltas = append(deltas, delta)
    if err := b.Delegate.Revert(
        sourcePath); err != nil {
      return err
    }
  }
  return b.Delegate.Write(
    resultPath, report(deltas))
}

type RealDelegate struct {
}

func Main() error {
  bench := Benchmark{Delegate: &RealDelegate{}}
  // these values would come from flags
  return bench.Benchmark(
    "binary", "source_path",
    "result_path", 3 /* numBuilds */)
}

// self-contained
func edit(content string, delta int) string {
  /* ... */
}

// self-contained
func report(deltas []time.Duration) string {
  /* ... */
}

// compiler
func (d *RealDelegate) Build(
    binary string) error {
  /* ... */
}

// file system
func (d *RealDelegate) Read(
    path string) (string, error) {
  /* ... */
}

// file system
func (d *RealDelegate) Write(
    path string, content string) error {
  /* ... */
}

// time
func (d *RealDelegate) Now() time.Time {
  return time.Now()
}

// version control
func (d *RealDelegate) Revert(
    path string) error {
  /* ... */
}

This makes it possible to create a FakeDelegate for the unit tests. But it is still a lot of work, specially in the Go language, to make this fake / mock / stub configurable to be able to control what it returns and perhaps also assert on the calls made.

Use Commands 🔗

Another way to make the original code more testable is to turn everything into commands that go through a single Runner:

type Runner interface {
  Run(
    command string,
    args ...string) (string, error)
}

type Benchmark struct {
  Runner Runner
}

func (b *Benchmark) Benchmark(
    binary string, sourcePath string,
    resultPath string, numBuilds int) error {
  if err := b.build(binary); err != nil {
    return err
  }
  source, err := b.read(sourcePath)
  if err != nil {
    return err
  }
  deltas := []float64{}
  for i := 0; i < numBuilds; i++ {
    if err := b.write(
        sourcePath,
        edit(source, i)); err != nil {
      return err
    }
    start, err := b.now()
    if err != nil {
      return err
    }
    if err := b.build(binary); err != nil {
      return err
    }
    end, err := b.now()
    if err != nil {
      return err
    }
    delta := end.Sub(start).Seconds()
    deltas = append(deltas, delta)
    if err := b.revert(sourcePath); err != nil {
      return err
    }
  }
  return b.write(resultPath, report(deltas))
}

type RealRunner struct {
}

func Main() error {
  bench := Benchmark{Runner: &RealRunner{}}
  // these values would come from flags
  return bench.Benchmark(
    "binary", "source_path",
    "result_path", 3 /* numBuilds */)
}

// self-contained
func edit(content string, delta int) string {
  return fmt.Sprintf("%s+%d", content, delta)
}

// self-contained
func report(deltas []float64) string {
  return fmt.Sprintf("%v", deltas)
}

// compiler
func (b *Benchmark) build(binary string) error {
  _, err := b.Runner.Run("build", binary)
  return err
}

// file system
func (b *Benchmark) read(
    path string) (string, error) {
  return b.Runner.Run("read", path)
}

// file system
func (b *Benchmark) write(
    path string, content string) error {
  _, err := b.Runner.Run("write", path, content)
  return err
}

// time
func (b *Benchmark) now() (time.Time, error) {
  out, err := b.Runner.Run("now")
  if err != nil {
    return time.Time{}, err
  }
  res, err := time.Parse(time.UnixDate, out)
  if err != nil {
    return time.Time{}, err
  }
  return res, nil
}

// version control
func (b *Benchmark) revert(path string) error {
  _, err := b.Runner.Run("revert", path)
  return err
}

func (r *RealRunner) Run(
    command string,
    args ...string) (string, error) {
  /* ... */
}

FakeRunner 🔗

This makes it much easier to create a fully configurable FakeRunner for the unit tests:

type FakeRunner struct {
  results []Result
  index   int
}

type Result struct {
  cmd string
  out string
  err error
}

func (f *FakeRunner) Run(
    command string,
    args ...string) (string, error) {
  if f.index >= len(f.results) {
    return "", fmt.Errorf(
      "Run %s %v index %d too high " +
      "for results %v", command, args,
      f.index, f.results)
  }
  res := f.results[f.index]
  f.index++
  got := join(command, args...)
  // would use diff.Diff
  if res.cmd != got {
    return "", fmt.Errorf(
      "Run %s %v got %s want %s",
      command, args, got, res.cmd)
  }
  return res.out, res.err
}

func join(
    command string,
    args ...string) string {
  return strings.Join(
    append([]string{command}, args...), " ")
}

func TestRun(t *testing.T) {
  inputs := []struct {
    results []Result
  }{{
    results: []Result{{
      cmd: "build binary",
    }, {
      cmd: "read source_path",
      out: "source_content",
    }, {
      // +0
      cmd: "write source_path source_content+0",
    }, {
      cmd: "now",
      out: "Mon Jan 01 00:00:00 MST 2000",
    }, {
      cmd: "build binary",
    }, {
      cmd: "now",
      // +10 seconds
      out: "Mon Jan 01 00:00:10 MST 2000",
    }, {
      cmd: "revert source_path",
    }, {
      // +1
      cmd: "write source_path source_content+1",
    }, {
      cmd: "now",
      out: "Mon Jan 01 00:00:10 MST 2000",
    }, {
      cmd: "build binary",
    }, {
      cmd: "now",
      // +5 seconds
      out: "Mon Jan 01 00:00:15 MST 2000",
    }, {
      cmd: "revert source_path",
    }, {
      cmd: "write result_path [10 5]",
    }},
  }}
  for _, in := range inputs {
    fake := &FakeRunner{results: in.results}
    bench := Benchmark{Runner: fake}
    if err := bench.Benchmark(
        "binary",
        "source_path",
        "result_path",
        2 /* numBuilds */); err != nil {
      t.Errorf(
        "Benchmark\n%#v\ngot err %v", in, err)
    }
    if fake.index != len(in.results) {
      t.Errorf(
        "Benchmark\n%#v\nfake.index %d " +
        "too low for len(in.results) %d",
        in, fake.index, len(in.results))
    }
  }
}

The main downside is that the code becomes less type-safe.

Typed Command Names 🔗

One way to make the code more type-safe is to use typed constants instead of strings for the command names:

type Command int

const (
  Build Command = iota
  Read
  Write
  Now
  Revert
)

var (
  commandNames = []string{
    "Build", "Read", "Write", "Now", "Revert"}
)

func (c Command) String() string {
  if int(c) < len(commandNames) {
    return commandNames[c]
  }
  return fmt.Sprintf("Command(%d)", c)
}

type Runner interface {
  Run(
    command Command,
    args ...string) (string, error)
}

type Benchmark struct {
  Runner Runner
}

func (b *Benchmark) Benchmark(
    binary string, sourcePath string,
    resultPath string, numBuilds int) error {
  /* ... */
}

type Fn func(args ...string) (string, error)

type RealRunner struct {
  Delegates map[Command]Fn
}

func Main() error {
  bench := Benchmark{Runner: &RealRunner{
    Delegates: map[Command]Fn{
      Build: func(
        args ...string) (string, error) {
          /* ... */
        },
      // ...
      // could check that
      // all commands are present
    },
  }}
  // these values would come from flags
  return bench.Benchmark(
    "binary", "source_path",
    "result_path", 3 /* numBuilds */)
}

// compiler
func (b *Benchmark) build(binary string) error {
  _, err := b.Runner.Run(Build, binary)
  return err
}

// file system
func (b *Benchmark) read(
    path string) (string, error) {
  return b.Runner.Run(Read, path)
}

// file system
func (b *Benchmark) write(
    path string, content string) error {
  _, err := b.Runner.Run(Write, path, content)
  return err
}

// time
func (b *Benchmark) now() (time.Time, error) {
  out, err := b.Runner.Run(Now)
  /* ... */
}

// version control
func (b *Benchmark) revert(path string) error {
  _, err := b.Runner.Run(Revert, path)
  return err
}

func (r *RealRunner) Run(
    command Command,
    args ...string) (string, error) {
  delegate, ok := r.Delegates[command]
  if !ok {
    return "", fmt.Errorf(
      "no delegate for command %s", command)
  }
  return delegate(args...)
}

Here it would also be possible to have each command be implemented in a different package and have each implementation register itself in the map of Delegates.