If there’s one thing doing formal methods has taught me, it’s that time is evil and I hate it. Time is just an inherently hard domain and there’s only so much languages can do to address concurrency.

Edit: successfully reproduced the bug in TLA+, verified that moving the channel receive to the worker fixes the deadlock.

Edit2: TLA+ spec is here

Give OTP a try sometime. Battle-hardened patterns for concurrency-model demands.

I would say especially in Go, since Go makes concurrency pretty hard compared to systems I’m used to…

Instead of spinning up one goroutine per task and rate-limiting the speed you create them, a cleaner solution is to spin up N goroutines and have them all read from a shared channel. Something like:

func forEachLimit(arr []int, fn func(int) err, parallelism int) err {
  queue := make(chan int)
  g := new(errorgroup.Group)

  for i := 0; i < parallelism; i++ {
    g.Go(func() {
      for item := range queue {
        if err := fn(item); err != nil {
          return err
        }
      }
      return nil
    })
  }

  for _, item := range arr {
    queue <- item
  }

  close(queue)

  if err := g.Wait(); err != nil {
    return err
  }

  return nil
}

FWIW, I do agree that concurrency in Go isn’t easy to get right. There was a paper a while ago where the authors found plenty of real world examples of deadlocks and memory leaks due to the specific behavior of channels. They also found Go programs tend to use way more concurrency than in other languages. I suspect this is because Go makes it so easy to write concurrent code, even when concurrency isn’t necessary.

Interesting tangental thought:

I saw this article and the URL and was like, “oh its the guy who hates Go again!” But when I looked at the actual website this author actually writes a significant amount about Go, the vast majority not negative. It seems like that the users of lobsters seem to be only interested in posting and upvoting anti-Go articles, and it is we who are biased.

Go can only provide low level concurrency tools because it’s statically typed. High level patterns require generics or dynamic typing so they can be used by applications. It’s between a rock and a hard place.

Related:

Channels Are Not Enough

An interesting argument that Go’s out-of-the-box concurrency abstractions are too low-level.

https://godoc.org/golang.org/x/sync/errgroup is very useful for this kind of thing.

Here is a pattern I’ve been using a lot lately which made concurrency much easier and safer for this kind of scenario:

func DoStuff(ctx context.Context, tasks []Something) (Result, error) {
	resultCh := make(chan Result)
	g, gCtx := errgroup.WithContext(ctx)

	// Can move this task provisioning block into a goroutine also, especially if
	// maintaining a max worker limit
	for _, t := range tasks {
		t := t // Copy value because we're passing it down the closure. A bit of a gotcha
		g.Go(func() error {
			// Usually bunch of code here, or just pass do_something into g.Go
			return do_something(t)
		})
	}

	// This is a bit awkward, I wish errgroup had something like:
	// g.Defer(func() { cleanup code that runs just before g.Wait() })
	gProcess, _ := errgroup.WithContext(gCtx)
	gProcess.Go(func() error {
		// This group closes resultCh which stops the accumulation loop below
		defer close(resultCh)
		return g.Wait()
	})

	var result Result
	for res := range resultCh {
		// accumulate result until resultCh is closed
	}

	return result, gProcess.Wait()
}

I’ve been using it so much that I’ve been thinking of upstreaming the errgroup.Group.Defer(func()) idea, or maybe maintaining a fork.

Go definitely provides better primitives than just locks for some concurrency patterns, but there are a lot of patterns that need to be built atop of them still, from what I can see. This looks like it could been useful to use something like a threadpool, or a higher order function for limiting the amount of parallelism (like a , though Go doesn’t make writing those sorts of higher level functions easy.

That being said, it’s definitely a subtle problem. Concurrency (via Channels or via Aysnc) is definitely something that can be easy to get tripped up on. Usually, I find that modeling problems by treating each part of the process as it’s own mini server can be handy, but supervision (which is necessary for systems over a certain amount of complexity) can still make things fiddly.