🐥 Mocking a HTTP server

If you are new to testing, you might want to read my post How I write Go Tests first.

When writing tests in Go, it's often to test the connection with an external API. Because your tests need to be consistent, you need to be sure that you test the connection to the API but also cover items such as invalid URLs to the API, connection timeouts, body read errors, …

In many cases, writing your tests this way makes testing faster as well as they are using a local webserver instead of a remote one. You basically rule out all possible errors which can occur when connecting to a remote server (DNS problems, network problems, …).

Therefor, it would be handy to be able to mock the webserver part for certain tests. The good news is that the testing package in Golang has this covered with the net/http/httptest package.

Let's start with a basic example on how we might want to test a simple API client which looks as follows:

package exampleapiclient

import (
    "io/ioutil"
    "net/http"
    "net/url"
    "time"
)

type APIClient struct {
    URL        string
    httpClient *http.Client
}

func NewAPIClient(url string, timeout time.Duration) APIClient {
    return APIClient{
        URL: url,
        httpClient: &http.Client{
            Timeout: timeout,
        },
    }
}

func (apiClient APIClient) ToUpper(input string) (string, error) {

    req, err := http.NewRequest("GET", apiClient.URL, nil)
    if err != nil {
        return "", err
    }

    q := req.URL.Query()
    q.Set("input", input)
    req.URL.RawQuery = q.Encode()

    resp, err := apiClient.httpClient.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    result, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }

    return string(result), nil

}

There are a few things in the design of the API client which will help up with testing:

• The URL is configurable so that we can override it during testing
• The timeout of the HTTP requests is also overridable

Testing a valid request

Let's start with testing a valid request:

package exampleapiclient_test

import (
    "net/http"
    "net/http/httptest"
    "strings"
    "testing"
    "time"

    exampleapiclient "github.com/pieterclaerhout/example-apiclient"
    "github.com/stretchr/testify/assert"
)

func TestValid(t *testing.T) {

    input := "expected"

    s := httptest.NewServer(
        http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
            w.Write([]byte(strings.ToUpper(input)))
        }),
    )
    defer s.Close()

    apiClient := exampleapiclient.NewAPIClient(s.URL, 5*time.Second)

    actual, err := apiClient.ToUpper(input)
    assert.NoError(t, err, "error")
    assert.Equal(t, strings.ToUpper(input), actual, "actual")

}

This is a very basic test which basically check if you are getting the expected response, but without relying on the live server.

The first thing we are doing here is to use httptest.NewServer to create a new server. Calling this allows you to define a handler function and also allows you to retrieve the URL to that server by using s.URL. You can use this for example to return static responses to check for example the parsing part of your package.

The next step is to create a new API client which connects to the URL of the server. That can be retrieved by s.URL. Then we execute and check that what we are getting the expected result.

Don't forget to call the defer s.Close() function to clean up after running the test.

Using testdata

The beauty of this approach is that any HTTP handler can be used. A neat trick is to store the expected data in the testdata folder and expose that to the HTTP server you have just created. Knowing that the working directory of your tests is the path of the package, you can do something as follows (provided you have a file testdata/index.txt which contains the string expected):

package exampleapiclient_test

import (
    "io/ioutil"
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestBasic(t *testing.T) {

    s := httptest.NewServer(
        http.FileServer(http.Dir("testdata")),
    )
    defer s.Close()

    resp, err := http.Get(s.URL + "/index.txt")
    assert.NoError(t, err, "error")

    defer resp.Body.Close()
    actual, err := ioutil.ReadAll(resp.Body)

    assert.Equal(t, expected, string(actual), "actual")

}

Testing invalid URLs

A next thing you probably want to check is how your package reacts when somebody passes in an invalid URL to the API. Before you can test this, yo need to ensure that your API package allows the user to override the URL.

func TestInvalidURL(t *testing.T) {

    apiClient := exampleapiclient.NewAPIClient("ht&@-tp://:aa", 5*time.Second)

    actual, err := apiClient.ToUpper("hello")
    assert.Error(t, err)
    assert.Empty(t, actual)

}

This test covers the fact that you might enter an invalid URL in the API client.

Testing read timeouts

Another thing you need to cover with your tests is that you are checking if you are properly handling time-outs in the API calls. That's why being able to set the timeout is important. A test can be written as follows:

func TestTimeout(t *testing.T) {

    s := httptest.NewServer(
        http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
            time.Sleep(50 * time.Millisecond)
            w.Write([]byte("actual"))
        }),
    )
    defer s.Close()

    apiClient := exampleapiclient.NewAPIClient(s.URL, 25*time.Millisecond)

    actual, err := apiClient.ToUpper("hello")
    assert.Error(t, err)
    assert.Empty(t, actual)

}

We are setting a timeout of 25 ms in the API client, but we are doing a sleep of 50 ms in the handler. If all goes well, the test should result in an error instead of a proper result.

Testing body read errors

The last thing you need to cover is what happens if there is a body read error. You can do this as follows:

func TestBodyReadError(t *testing.T) {

    s := httptest.NewServer(
        http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
            w.Header().Set("Content-Length", "1")
        }),
    )
    defer s.Close()

    apiClient := exampleapiclient.NewAPIClient(s.URL, 25*time.Millisecond)

    actual, err := apiClient.ToUpper("hello")
    assert.Error(t, err)
    assert.Empty(t, actual)

}

The trick is to send an invalid Content-Length. This causes a body read error.

This way, we have the complete package covered:

$  go test -cover
PASS
coverage: 100.0% of statements
ok      github.com/pieterclaerhout/example-apiclient    0.077s

You can find the complete example app here.