Web UI Testing With Go

Not using Cypress, Selenium or Playwright

Inkmi is Dream Jobs for CTOs and written as a decoupled monolith in Go, HTMX, Alpinejs, NATS.io and Postgres. I document my adventures and challenges in writing the application here on this blog, tune in again.

When working on my new startup, I knew testing was essential to scale. As a solo entrepreneur, tests are even more important for confident releases. Without confident releases, you either don’t release as often as possible or you have a bad feeling each time. Or you have broken releases.

Therefore, from the beginning, I’ve added UI browser tests with Playwright. At least having the core flows under test, like login, registration, matching and onboarding. This way I could release with confidence and be sure no crucial flow is broken.

Inkmi - Dream Jobs for CTOs is written in Go. I love Go for it’s simplicity. We’re still after some years in the honeymoon phase. Inkmi is a server application that renders HTML to the browser, not an SPA (because as a solo entrepreneur I have to be frugile with my time). I didn’t find a good browser driver in Go so I needed to write tests in JS/Typescript. No problem, I have been coding Javascript/TS for several decades now. But cognitive load grew, no way around it. Overall Playwright is nice, documentation is great, there are many examples and a nice runner that takes screenshots on a timeline. I got fast to running tests.

After some weeks, I found problems with UI browser testing. Creating a starting data set in the database was a challenge, as I could only create data directly in the database from Javascript, not use the Golang business logic (or I would need to add an additional REST API to the backend). This often misses crucial side effects. Setting up data through the browser tests was prohibitively slow. In startups and larger companies, I worked UI tests were slow and took a long time to run, so they were seldom run. I had to make them as fast a possible to want to run them often.

(Sadly they were still not as fast as I had wished)

Third, the Playwright tests were flaky. They worked, then with no code changes they broke due to race conditions of a complex browser environment. Adding waiting some time, and waiting for browser events helped somehow, but slowed them down and never made the flakiness go away completely.

After wasting a whole day of trying to fix some flakiness, I gave up and deleted all of my UI browser tests in an instant!

What to do now? I had good experience with Golang E2E tests for checking server work flows. They were fast, I could test business logic end to end, and they were deterministic.

Why not do web testing directly with Go? Inkmi doesn’t use a lot of JS, so I might not need the whole browser environment. Inkmi only uses some Alpine.js for dropdowns and HTMX for form submissions. I don’t test the Alpine.js parts (for now), and I wrote some custom code to work with HTMX (merging not tested).

The basic idea is I use Go HTTP client to get a page, then use goquery to parse the result and check for the title, other metadata and data in the body. If there is a form, I extract the form URL from the page and call the URL with JSON data. Again I parse the result and so on. Everything encapsulated in one flow, e.g. Login flow.

(Simplified)

// Create a page struct from an URL with the builder `HomePageFrom`
// c is a context object that keeps CSRF and session tokens between
// calls
homePage := FromUrl(t, "/", HomePageFrom, c)
// use test assertions to check the homepage
// is correct and has the data as expected
homePage.Check(t)

// We use the url from the homepage to go to the login page
loginPage := FromUrl(t, homePage.LoginUrl, LoginPageFrom, c)
loginPage.Check(t)

// Use JSON data to login
loginData := LoginData{
Email: email,
}
// Post to the HX-Post URL from the login Page
inbox := PostToUrl(t, loginPage.HxPost, loginData,
InboxPageFrom, c)
inbox.Check(t)

For login I sent emails with a link. How to keep the flow going when it spawns into mails?

A helper can get Mails that are sent out. Then a builder just like the page builder builds a mail struct from the HTML in the mail and extracts needed data.

WaitForMail("Your link to login to Inkmi", email)
loginMail := mails.FromMail(t,
"Your link to login to Inkmi",
email,
mails.LoginMailFrom,
a,
)
loginMail.Check(t)

When we expect a redirect there is a helper for that too:

redirect := RedirectFromUrl(t, loginMail.LoginUrl, c)
assert.NotEmpty(t, redirect)

The UI tests are standard Go tests. To only run tests when I want them to run, not on every go test I use build tags.

//go:build long_tests
package htmltest

These tests then are only run with gotestsum -- -tags=long_tests ./...

Voila!

Results

The results are great. The tests are much faster and not flaky at all but deterministic. They are written in Go, so I don’t have to switch languages. The tests can easily use business logic to create data for test setup and check the database after running tests, again by using business logic instead of plain SQL.

Downsides? You need to be always aware of what you are really testing. The coverage is not the same as with Playwright. I don’t simulate a browser with all its technologies and environment. I don’t test with different browsers like Chrome or Safari—obviously.

JS could be broken, and the HTML forms could be broken. Currently, I directly post JSON to the POST URL. In the future, it should be safer to fill the HTML form with goquery

// uses goquery to change the DOM/HTML of the form
loginPage.FillEmail(email)

and then let the JS code run on it (perhaps with Otto).

// extracts the data with JS/otto from the DOM and creates
// a map to use with POST
loginData := loginPage.PostData()

But this might be overengineering. I’ll start when I have the first broken release.

This doesn’t work for everyone—but might be the right way for server-HTML/HTMX projects.

Overall I’m very happy.

About Inkmi

Inkmi is a website with Dream Jobs for CTOs. We're on a mission to transform the industry to create more dream jobs for CTOs. If you're a seasoned CTO looking for a new job, or a senior developer ready for your first CTO calling, head over to https://www.inkmi.com