A Useful Pattern for Nil Channel Values in Go
At DoltHub we're building Dolt, a SQL database that supports Git-like version control, including branch, merge, diff, clone, push, and pull. Dolt is written in Go. This post is part of a series of blog posts about programming in Go.
I first started working with Go as my primary programming language over 6 years ago when we started DoltHub. Go is a practical language with a great library ecosystem and I found it relatively easy to get up to speed. That being said, there were a few gotchas and sharp edges to which I had to acclimate. We've written about a few in previous blog posts.
One thing that was slightly confusing to me when I first started with Go was the behavior of channel send and receive on a nil
-valued channel variable. For both send and receive, if the channel value is nil
, the calling thread blocks indefinitely. In isolation, this just seemed like a way-too-easy-way to deadlock a goroutine.
As a quick reminder, here is the behavior of send and receive on various channels in Go:
Case | Send Beahvior | Receive Behavior |
---|---|---|
Open, unbuffered | Block until another goroutine receives the value | Block until another goroutine sends a value |
Closed | Panic | Immediately receive a zero value. Optionally receive an indication that the channel is closed |
Nil | Block forever | Block forever |
With further reflection and experience, I realized that the nil
behavior of channels in Go enables a number of useful and idiomatic patterns. Instead of focusing on the behavior in the context of a single-channel send, receive or range statement, the fundamental use case for the nil
channel behavior is to selectively disable a specific branch of a select
statement. Let's take a look a few useful patterns.
Optional Functionality
If your select statement has optional functionality, you can sometimes keep a single select statement to implement the functionality by making use of a channel variable which is optionally nil
. For example, if we want to receive from a channel while enforcing an optional timeout, we can do something like:
var timeout chan time.Time
if enforceTimeout {
timeout = time.After(10 * time.Second)
}
select {
case recv := <-recvCh:
return recv, nil
case <-timeout:
return nil, errors.New("timed out")
}
It easily works in the send case as well. For example, if we want to deliver the same value to three different channels:
for i := 0; i < 3; i++ {
select {
case chans[0] <- val:
chans[0] = nil
case chans[1] <- val:
chans[1] = nil
case chans[2] <- val:
chans[2] = nil
}
}
Of course, it's also possible to spawn three goroutines to deliver the value to the three separate channels. But if you need to coordinate on the event of all three being sent, you need to bring another synchronization primitive, like a wait group or another channel, into the mix. The nil
channels might be simpler, slightly more efficient, and easier to read. For comparison, the goroutine solution might be implemented as:
var wg sync.WaitGroup
wg.Add(3)
for i := 0; i < 3; i++ {
i := i
go func() {
defer wg.Done()
chans[i] <- val
}()
}
wg.Wait()
State Machines, Input / Output Branches
The real benefit in expressive power can be seen in more complicated use cases where various branches of a select statement, usually continually visited in a loop, need to be enabled or disabled depending on the local state of the goroutine. For example, if we have an input channel and we want to send along its latest emitted value, but only every so often, we can do something like:
// Emit the latest value from |inputCh| onto |outputCh| once every |inbetween|,
// or once there is a new value to emit if the time between the values is
// larger than |inbetween|. Starts delivering as soon as the first value comes
// in from |inputCh|
func Debounce[T any](inputCh <-chan T, outputCh chan<- T, inbetween time.Duration) {
var latest T
var curOutCh chan<- T
var timeoutCh <-chan time.Time
canSend, hasNewValue := true, false
for {
if hasNewValue && canSend {
curOutCh = outputCh
} else {
curOutCh = nil
}
select {
case in, closed := <- inputCh:
if closed {
return
}
latest = in
hasNewValue = true
case <-timeoutCh:
canSend = true
timeoutCh = nil
case curOutCh <- latest:
canSend, hasNewValue = false, false
timeoutCh = time.After(inbetween)
}
}
}
Another instructive example is implementing a batching channel transformer with a maximum size. The transformer takes a chan T
and converts it to a chan []T
, where the delivered slice has a maximum size. The transformer is always willing to immediately deliver a smaller batch, but it avoids buffering too much data if the receiver has lower throughput than the sender. The code enables the output channel when it has anything to send and it disables the input channel once the batch is full:
func Batch[T any](inputCh <-chan T, outputCh chan<- []T, sz int) {
var batch []T
for {
if batch == nil && inputCh == nil {
return
}
var localOutCh chan<- []T
var localInCh <-chan T
if len(batch) > 0 {
localOutCh = outputCh
}
if len(batch) < sz {
localInCh = inputCh
}
select {
case input, closed := <-localInCh:
if !ok {
inputCh = nil
} else {
batch = append(batch, input)
}
case localOutCh <- batch:
batch = nil
}
}
}
These are just a few of the simpler examples of interesting behavior that can be built making use of the nil
-channel send and receive behavior.
Conclusion
And so it can be seen that this initially counterintuitive beahvior has a compelling use case when it comes to select
statements and optionally enabling and disabling branches. Lots of useful behavior can be implemented by optionally disabling branches at certain times when they don't apply, and the code is simpler than having separate select
statements for all the variations of enabled and disabled channels. Have you come across this pattern in your own code? Reach out to us on Discord to discuss nil
-valued channel behavior, Go in general or SQL servers with branch, merge and diff.