Integration tests for a wallet, with the Robot pattern

Context

In mid 2024 Cake Wallet had unit tests and human testers, and nothing in between. Every release meant someone creating a wallet, restoring one, checking the seed, sending, exchanging and reading history by hand, on two platforms.

I built the integration test suite between July 2024 and July 2025: four merged changes, about 7,800 lines added and 1,900 removed, across 227 file touches. It is still in the tree and three of its flows run on every pull request.

Problem

Wallet flows are long. Creating a wallet is nine screens. The screens change often, because the app was being redesigned underneath the tests the whole time. And the two flows that matter most, send and exchange, need a live provider and a wallet with coin in it, which a test runner does not have.

Constraints

Tests that find widgets by text break on every copy change and cannot tell two identical cards apart.

Seeds for test wallets can never be in the repo. They arrive at CI time from secrets.

Emulators in CI are mostly not about tests. They are permissions, boot readiness, animation scales, snapshot caching and memory fights between the build and the emulator on the same runner.

Decisions and trade-offs

One shared foundation, not one branch per flow (September 2024). Four changes were open at once in August 2024, three of them a flow each. I landed the shared foundation first and closed the other three five days later, re-landing their flows on top of it. The cost was throwing away roughly 3,600 lines of already-written work. What it bought is the reason the pattern survived: one file, the common test cases, is the only place that knows Flutter’s test API, so a framework change lands in one file, not 26.

One robot per screen. A robot exposes three kinds of method and nothing else: an identity check for the page, assertions about what the page should show, written against the app’s own localisation strings and keys rather than literal text, and actions a person can take. A robot can hold another robot when one screen can interrupt another; the send robot holds the auth robot because the lock screen can appear over the send flow. Flows compose robots into journeys, so a flow suite reads as a short list of steps.

Keys on production widgets rather than finding by text. 255 value keys across the first two changes, added to shipping widgets. It couples the app to its tests and shows up in every later diff. It is also the only version that survives copy changes and can address two identical cards. Where a widget renders twice, the key is built from an instance name.

Make the debug skip CI-aware instead of deleting it (July 2025). Seed verification is skipped in debug so developers are not retyping seed words all day. I was going to remove the skip so the test exercised the step. A reviewer pointed out that would make local work worse. The landed version keys the skip off debug mode and not CI, so local behaviour is unchanged and the step runs in CI and release.

An empty test wallet is not a test failure. The send suite tries three wallets in order and only sends from the first with enough balance. If none has any, it says so and passes. The alternative, a red build every time a faucet runs dry, gets the suite ignored. The cost is that a broken send path whose error mentions balance could be misread as an empty wallet; the open rework adds a classifier that separates empty, waiting and failed.

Keep the test that moves money out of the discovered directory. One test restores a funded wallet and runs a real exchange. It lives outside the suites folder so the runner never picks it up by accident, and the amounts in the constants file are tiny.

What was hard

Long changes against a moving app. 81 days, 50, 66 and 182. Each one touches dozens of app files, so each one rebased against a codebase being redesigned underneath it.

Flaky by default. The runner grew retries, per-test timeouts, an inactivity ceiling and data cleanup between attempts. One page-title assertion failed intermittently in a way that looked like a framework issue rather than a product one, and I said so in review rather than papering over it.

Keeping it alive. The Android emulator workflow runs on demand today, not on every push. The coverage that survived on the pull-request gate is the Linux desktop run: three of the six flows, on every pull request unless the commit opts out, with the screen recorded and uploaded. The open rework is the attempt to put the Android side back on the gate, with funds tests behind a manual workflow that defaults to a dry run.

Result

Six flow suites: create a wallet and verify its seed, restore from a seed, confirm the credentials shown match the wallet, send, exchange, and read transaction history. Twenty-six screen robots. One command runs all six against an Android emulator or a desktop build with retries and a summary, and I run them on Android and iOS locally. Three run in CI on every pull request.

No pass rate or bug count appears here because the repo does not record one, and I would rather say what runs than guess what it caught.