Go 1.18 has been out for a while now. Are you using Generics?

At first, I was delighted, started gradually switching from map and filter-like for statements to github.com/samber/lo and I created an internal slice package to add some more variants.

However, somewhere in my heart, I felt frustration building up. I felt like I was missing out on something like I didn’t understand its true value because I hadn’t used it to its fullest. It is as if I have been given such an interesting toy and have not played with it to the fullest.

I want to write code using Generics!

I’ve been writing REST APIs in my daily job, and I’ve been thinking … isn’t this is the place in the conclusion of When To Use Generics:

If you find yourself writing the exact same code multiple times, where the only difference between the copies is that the code uses different types, Consider whether you can use a type parameter.

Aren’t we writing the same code many times?
Aren’t we writing REST APIs over and over again?
Isn’t it time to use Generics to build REST APIs?

I thought I’d give it a shot, and have experimented with Generics for a few days. The results are as follows.

Ghost - Build REST APIs from structs using Generics

The README of the package github.com/mash/ghost contains the following example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

type User struct {
  Name string
}

type SearchQuery struct {
  Name string
}

func main() {
  store := ghost.NewMapStore(&User{}, SearchQuery{}, uint64(0))
  g := ghost.New(store)
    // g is a http.Handler and it provides GET /?name=:name, GET /:userid, POST /, PUT /:userid, DELETE /:userid
    ListenAndServe("127.0.0.1:8080", g)
}

Can you see the intent of this?
User type is a resource, that Ghost turns into an HTTP handler that provides POST, GET, PUT, and DELETE HTTP methods calls for CRUD operations.
SearchQuery type represents the search query for GET /.
User resources have an uint64 identifier, which constructs the path for the REST API.

When you hear the word “REST API,” you may have a variety of different styles in mind. The request and response bodies may be JSON, MessagePack or Protocol buffers etc. When returning an error, will it contain an error code and a human-readable msg or do you prefer “error_description”?

The constraints should be as loose as possible to allow for various styles of implementation. I tried to construct Ghost as a set of simple generic interfaces so that package users can implement their style and be able to swap and combine the generic interfaces to build the REST API.

Three generic types are used. In this example, a resource of type User is type Resource any, a list query of type SearchQuery is type Query any, and an identifier of type uint64 is type PKey interface { comparable } as the type constraint.

…

Here are some of my findings that I found interesting while writing this.

1. Generic interfaces to escape the generic world

One of my favorite design patterns when writing Go is the Middleware Design Pattern.

By providing a standard interface that inserts operations before and after an operation at the center of an onion, simple-functional middlewares (layers of onions) can be combined to describe complex operations as a whole. We see this a lot in http.Handler. (PSGI/Plack was probably the first one that shocked me when I first saw it).

I thought it would be effective to apply the same pattern to REST API, so I defined the following generic interface.

1
2
3
4
5
6
7

type Store[R Resource, Q Query, P PKey] interface {
  Create(context.Context, *R) error
  Read(context.Context, P, *Q) (*R, error)
  Update(context.Context, P, *R) error
  Delete(context.Context, P) error
  List(context.Context, *Q) ([]R, error)
}

Let’s start with an obvious feature for APIs. Request parameters validation. I implemented this interface and used github.com/go-playground/validator to validate request parameters before CRUD operations, and that’s github.com/mash/ghost/store/validator. An excerpt is shown below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

type validatorStore[R ghost.Resource, Q ghost.Query, P ghost.PKey] struct {
  store ghost.Store[R, Q, P].
  validate *validator.Validate
}

func NewStore[R ghost.Resource, Q ghost.Query, P ghost.PKey](store ghost.Store[R, Q, P], validator *validator.Validate) ghost.Store[R, Q, P] {
  return validatorStore[R, Q, P]{
      store: store,
      validate: validator,
  }
}

func (s validatorStore[R, Q, P]) Create(ctx context.Context, r *R) error {
  if err := s.validate.StructCtx(ctx, r); err ! = nil {
      return validationError(err)
  }
  return s.store.Create(ctx, r)
}

Simple and good.

We often see cases when you just can’t express your validation rules in Golang tags. It would be nice to be able to call hooks before CRUD operations to implement custom validations. So I created hookStore.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

type hookStore[R Resource, Q Query, P PKey] struct {
  store Store Store[R, Q, P]
}

func NewHookStore[R Resource, Q Query, P PKey](store Store[R, Q, P]) Store[R, Q, P] {
  return hookStore[R, Q, P]{
      store: store,
  }
}

type BeforeCreate interface {
  BeforeCreate(context.Context) error
}

} type AfterCreate interface {
  AfterCreate(context.Context) error
}

func (s hookStore[R, Q, P]) Create(ctx context.Context, r *R) error {
  if h, ok := any(r). (BeforeCreate); ok {
      if err := h.BeforeCreate(ctx); err ! = nil {
          return err
      }
  }
  if err := s.store.Create(ctx, r); err ! = nil {
      return err
  }
  if h, ok := any(r). (AfterCreate); ok {
      if err := h.AfterCreate(ctx); err ! = nil {
          return err
      }
  }
  return nil
}

The argument r of generic type *R, can be cast to any®and then we can use type assertion to check if r implementsBeforeCreate,AfterCreate` interfaces. This is familiar, you can do this without generics.
But how about …

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

type BeforeList[Q Query] interface {
  BeforeList(context.Context, *Q) error
}

type AfterList[R Resource, Q Query] interface {
  AfterList(context.Context, *Q, []R) error
}

func (s hookStore[R, Q, P]) List(ctx context.Context, q *Q) ([]R, error) {
  var r R
  if h, ok := any(&r). (BeforeList[Q]); ok {
      if err := h.BeforeList(ctx, q); err ! = nil {
          return nil, err
      }
  }
  l, err := s.store.List(ctx, q)
  if err ! = nil {
      return l, err
  }
  if h, ok := any(&r). (AfterList[R, Q]); ok {
      if err := h.AfterList(ctx, q, l); err ! = nil {
          return nil, err
      }
  }
  return l, nil
}

I discovered that you can also type assertion to generic interfaces BeforeList[Q] and AfterList[R, Q] with R and Q as type parameters! It’s starting to look like something I haven’t seen anywhere else.

The code to implement these interfaces is in the test.

1
2
3
4
5
6
7
8
9

func (u *HookedUser) BeforeList(ctx context.Context, q *SearchQuery) error {
  globalCalled["BeforeList"]++ // record the call for testing
  return nil
}

func (u *HookedUser) AfterList(ctx context.Context, q *SearchQuery, rr []HookedUser) error {
  globalCalled["AfterList"]++
  return nil
}

This is interesting. I can call a function with a non-generic concrete type as an argument from within the generic world. To avoid typos and accidentally not implementing the interface, it is safe to write as follows.

1
2

var _ ghost.BeforeList[SearchQuery] = &HookedUser{}
var _ ghost.AfterList[HookedUser, SearchQuery] = &HookedUser{}

I also created gormStore using the OR mapper, gorm and applied the same mechanism to implement hooks. I wrote gormStore so that if the resource struct implements the generic interface, that overrides the generic implementation. Take a look if you are interested.

2. Specialization?

I have only written C like C++ and have avoided Java as much as possible, so my vocabulary for Generics is lacking. It’s hard to explain why I find this interesting, but let’s go ahead.

If you run the above example, you will get a path named /:userid, where the :userid part is a uint64. Since the path is a string, the string is converted to uint64 by strconv.ParseUint.

When you create a REST API, you may want to use a string (often called a slug) as an identifier, like the maaash part of twitter.com/maaash. However, there is no need to strconv.ParseUint a slug.

So I implemented each separately.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

type PKey interface {
  comparable
}

type PUintKey interface {
  comparable
  ~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64
}

type PStrKey interface {
  comparable
  string
}

type Identifier[P PKey] interface {
  PKey(*http.Request) (P, error)
}

// PathIdentifier is an Identifier which extracts the PKey from the request URL path.
type PathIdentifier[P PKey] func(string) (P, error)

func (pi PathIdentifier[P]) PKey(r *http.Request) (P, error) {
  var p P
  _, lastpath := path.Split(r.URL.Path)
  if lastpath ! = "" {
      return pi(lastpath)
  }
  return p, ErrNotFound
}

func UintPath[P PUintKey](s string) (P, error) {
  i, err := strconv.ParseUint(s, 10, 64)
  return P(i), err
}

func StrPath[P PStrKey](s string) (P, error) {
  return P(s), nil
}

As I wrote in the beginning,

I tried to construct Ghost as a set of simple generic interfaces so that package users can implement their style and be able to swap and combine the generic interfaces to build the REST API.

Identifier is one such component, a generic interface to retrieve a PKey from an HTTP request. PathIdentifier is one of its generic implementations, which retrieves PKey from URL.Path of the HTTP request. UintPath and StrPath handle the case where the PKey is a uint or a string, respectively.

I find that fascinating that … The whole “ghost” uses a loose constraint that is type PKey interface { comparable }, while the PUintKey and PStrKey, which are stricter variants of PKey, are used to cover the most major use cases. If package users want to use their special type, they can implement the Identifier interface themselves, and it can be integrated into Ghost…

…

If you have reached this point, you may think I am overusing generics. I honestly agree too. Some may criticize that you can achieve the same with interfaces. Have you read the intro? The whole point of this article is to try to use the hammer. But on the other hand, what if there were more ready-made generic implementations of the components in Ghost?

I’m thinking of using it in a side project first.

I’m glad that I made Ghost because it relieved my frustration of not being able to fully touch Generics. I was worried that the inclusion of generics in Go 1.18 would make the code more difficult to read, but I’m grateful to the team for how it landed. Go is continuing to be really easy to read.

My biggest complaint about Generics in Go is … Github Copilot is not very helpful anymore. Seriously, if the accuracy of the code proposed by Copilot is poor, my productivity will decrease. Let’s all write public code using Generics and feed Copilot!

If you find it interesting, I will be glad if you give me feedback in the repository or @maaash](https://twitter.com/maaash).

This is an English translation of the original Japanese article that I wrote on Golang GenericsでREST APIを作る.