Error Handling in Go

[sudo-suhas]

Proposing some changes to the error handling practices as part of this discussion.

Links

Repo: https://github.com/sudo-suhas/xgo
Usage docs: https://github.com/sudo-suhas/xgo/tree/master/errors
API Reference: https://pkg.go.dev/github.com/sudo-suhas/xgo/errors

The basics

To facilitate error construction, the package provides a function, errors.E that takes at least 1 constructor option:

func E(opt Option, opts ...Option) error

And this is how it may look in the most common case:

if err := svc.ProcessSomething(); err != nil {
    return errors.E(errors.WithOp(op), errors.WithErr(err))
}

You can optionally pass in more than 1 constructor option to associate additional data with the error.

What are the problems?

The ubiquitous way of creating errors with errors.New or wrapping errors with fmt.Errorf have inherent limitations:

• You can only associate a string value with the error. This results in multiple problems.
  • Not every information you want to associate with the error will be best represented as a string. For example, it might be better if we could associate and pass the JSON HTTP response as a json.RawMessage in an error. This can later be logged in a structured format.
  • Bloated error strings. The error string has to always do a balancing act between being concise and retaining appropriate failure context.
  • Often not all the error context can be passed back up and the information is logged instead. This results in multiple log statements for a single error that is tedious and difficult to link together the different logs to be able to fully understand what went wrong.
• Being able to handle errors necessitates the declaration of sentinel errors in most cases. And the handling will grow linearly with the sentinel errors being declared. It makes it difficult to introduce automated handling where every error, irrespective of the endpoint, can be run through a single function to determine the response that we want to return to the client.

How does it solve the problems?

Associating error information

The library broadly allows for associating the following optional information with the error:

• errors.WithOp: The event or operation resulting in an error, most commonly represented by pkgname.FunctionName. This builds up to the sequence of logical steps, a logical stack trace, if you will. This is also important because the same leaf function can be called from multiple paths and being able to capture the sequence is quite useful for debugging.
• errors.Kind: This type implements the errors.Option interface and can be passed in as a constructor option directly. The kind will determine the classification of the error, ex: permission error, timeout error etc. The responsibility of deciphering the failure from a lower level API is best left to the layer that integrated it. The same can be inspected using errors.WhatKind.
• errors.WithText/ errors.WithTextf: Arbitrary string to add context to the error.
• errors.WithUserMsg: A message to be returned back to the user. The error message originating from libraries is not appropriate for showing to the user. This optional constructor option can be used to associate the user message we want to return at the service level. The same message can then be retrieved using errors.UserMsg while trying to construct the response for the caller.
• errors.WithData: Associate arbitrary data with the error. Example: In errors.WithResp, this is used to add the JSON response to the error without converting it into a string.

An example:

// CreateUser creates a new user in the system.
func (s *Service) CreateUser(ctx context.Context, user *myapp.User) error {
    const op xgo.Op = "svc.CreateUser"

    // Validate username is non-blank.
    if user.Username == "" {
        msg := "Username is required"
        return errors.E(errors.WithOp(op), errors.InvalidInput, errors.WithUserMsg(msg))
    }

    // Verify user does not already exist
    if s.usernameInUse(user.Username) {
        msg := "Username is already in use. Please choose a different username."
        return errors.E(errors.WithOp(op), errors.AlreadyExists, errors.WithUserMsg(msg))
    }

    // ...
}

Being able to associate these types of information allows us to do the following:

• Log the error only once using *error.Error.Details() since it has all the relevant information, preferably near the end of processing for a request or operation.
• Return an appropriate and friendly error message to the client without exposing implementation details.
• Have a single way of handling all errors irrespective of the handler.

Handle errors based on attributes

Rather than writing code dependent on specific error values or types, it would be better to rely on the classification or kind of error. In the majority of cases, we would pick the code branch to execute based on the classification of error. As mentioned above, the kind can be passed in as a constructor option. Inspecting the error is quite straightforward:

if errors.WhatKind(err) == errors.NotFound {
    // ...
}

Additionally, more often than not, the type of error at the site of construction determines the response being sent back to the client. And the site at which the error originated is the most qualified to identify and associate the kind of the error. For example, let's say you tried to fetch a resource from your database as part of a GET call but it was not found. Your response is likely going to be 404 or an equivalent. In such a case, based on the sentinel error defined in the database's package, the kind could be set as errors.NotFound.

Sugar

Additional cherries on top.

Hook into the constructor

You can write custom constructor options that can take arbitrary data and translate that into 1 or more fields for the error.

func withSQLError(err error) errors.Option {
    return errors.OptionFunc(func(e *errors.Error) {
        if err == sql.ErrNoRows {
            err.Kind = errors.NotFound
        }

        e.Err = err
    })
}

Here is another example for parsing information from a HTTP response while constructing the error - errors.WithResp.

Interop with errors.{Is, As}

The Error type implements the interface required by the standard library's Unwrap method - xgo@v0.2.0/errors/error.go#L110. This means that wrapped errors can still be checked with errors.{Is, As} if required.

Bring your own kind

Apart from the error kinds declared in the library it is possible and encouraged to declare custom ones if and when needed. An example, as seen here:

var (
    ErrKindUnsupportedMediaType = errors.Kind{
        Code:   "UNSUPPORTED_MEDIA_TYPE",
        Status: http.StatusUnsupportedMediaType,
    }
    ErrKindRequestEntityTooLarge = errors.Kind{
        Code:   "REQUEST_ENTITY_TOO_LARGE",
        Status: http.StatusRequestEntityTooLarge,
    }
)

Bring your own error

The error package defines and uses interfaces/traits such as errors.GetKind, errors.StatusCoder to allow interoperability with custom error definitions. This means that while using the errors package, you aren't locked into the types declared by it; you can declare and use custom ones if required.

HTTP Interop

xgo@master/errors#http-interop

Tell it how to JSON

There is a default implementation provided for converting the error into a serializable struct - xgo@v0.2.0/errors/err_json.go#L19. But this can be overridden by using errors.WithToJSON. An example:

func makeErrToJSON(lang string) errors.JSONFunc {
    return func(e *errors.Error) interface{} {
        msgKey := errors.UserMsg(e)
        return map[string]interface{}{
            "code":    errors.WhatKind(e).Code,
            "title":   i18n.Title(lang, msgKey),
            "message": i18n.Message(lang, msgKey),
        }
    }
}

// In the HTTP handler, when there is an error, wrap it and pass it to the
// response formatter.
errors.E(errors.WithToJSON(makeErrToJSON(lang)), errors.WithErr(err))

What's missing

• Good examples showing how it could be integrated at different layers of a service/application.
• Blog posts to make it easier to get started with the library.
• The library provides some interop with HTTP and clearly takes inspiration from the error codes defined in gRPC. But as of today, it does not have any helpers in place to make it easier to work with gRPC.
• Ability to transcode the error across a network connection during RPC calls.

Credits

See https://github.com/sudo-suhas/xgo#credits.