End-to-end functional tests

Scripted user journeys through the real UI against a deployed build.

Mandatory for 43 of 46 product types. Usually run by a QA engineer.

What it catches

What it misses

Which product types need it and why

Status of end-to-end functional tests on every published playbook, with the risk ids or the condition behind it.
Product typeStatusReason or condition
accounting and invoicing SaaSmandatoryRisks: Fraud or abuse by users
buy now pay later servicemandatoryRisks: Fine or enforcement by a regulator
clinical trial management softwaremandatoryRisks: Physical or psychological harm to a user
crypto exchangemandatoryRisks: Fraud or abuse by users, Money leaves or fails to arrive
dating appmandatoryRisks: Fraud or abuse by users, App rejected or removed by a store
desktop applicationmandatoryRisks: Records silently wrong, Personal or sensitive data exposed
e-signature servicemandatoryRisks: Fine or enforcement by a regulator
ecommerce storemandatoryRisks: Fine or enforcement by a regulator, Money leaves or fails to arrive
electronic health record systemmandatoryRisks: Physical or psychological harm to a user
email marketing platformmandatoryRisks: Fine or enforcement by a regulator, App rejected or removed by a store
ERP systemmandatoryRisks: Money leaves or fails to arrive
EV charging appmandatoryRisks: Users or operators act on wrong numbers, Fine or enforcement by a regulator
expense management toolmandatoryRisks: Fine or enforcement by a regulator, Money leaves or fails to arrive, Fraud or abuse by users
fleet and logistics trackingmandatoryRisks: Physical or psychological harm to a user, Fine or enforcement by a regulator
government services portalmandatoryRisks: Fraud or abuse by users
grocery delivery appmandatoryRisks: Fine or enforcement by a regulator
identity and access management servicemandatoryRisks: Fraud or abuse by users
insurance appmandatoryRisks: Personal or sensitive data exposed
smart home or IoT appmandatoryRisks: Fraud or abuse by users, Physical or psychological harm to a user
kids appmandatoryRisks: Money leaves or fails to arrive, Physical or psychological harm to a user
language learning appmandatoryRisks: Money leaves or fails to arrive, Fine or enforcement by a regulator
lending platformmandatoryRisks: Users or operators act on wrong numbers, Fine or enforcement by a regulator
LLM-based assistantmandatoryRisks: Physical or psychological harm to a user
loyalty program appmandatoryRisks: Money leaves or fails to arrive, Fine or enforcement by a regulator
online marketplacemandatoryRisks: Fraud or abuse by users, Fine or enforcement by a regulator
mental health appmandatoryRisks: Money leaves or fails to arrive, Fine or enforcement by a regulator
messaging appmandatoryRisks: Physical or psychological harm to a user
mobile banking appmandatoryRisks: Fraud or abuse by users, Money leaves or fails to arrive
mobile gamemandatoryRisks: Physical or psychological harm to a user, Fine or enforcement by a regulator
neobankmandatoryRisks: Fraud or abuse by users, Money leaves or fails to arrive
peer-to-peer payment appmandatoryRisks: Money leaves or fails to arrive, Fraud or abuse by users
patient portalmandatoryRisks: User actions lost on bad networks or devices, Fine or enforcement by a regulator, Personal or sensitive data exposed
pharmacy delivery appmandatoryRisks: Fine or enforcement by a regulator, App rejected or removed by a store
ride-hailing appmandatoryRisks: Accessibility complaint or lawsuit
software as a medical devicemandatoryRisks: Physical or psychological harm to a user
social networkmandatoryRisks: App rejected or removed by a store
subscription box servicemandatoryRisks: Fine or enforcement by a regulator
telemedicine appmandatoryRisks: Physical or psychological harm to a user, Fine or enforcement by a regulator
event ticketing platformmandatoryRisks: Fine or enforcement by a regulator, Fraud or abuse by users
stock trading appmandatoryRisks: Money leaves or fails to arrive
travel booking appmandatoryRisks: Fine or enforcement by a regulator
video conferencing toolmandatoryRisks: Fraud or abuse by users
wearable companion appmandatoryRisks: Physical or psychological harm to a user
SDK or client libraryoptionalOptional: catches doc drift, no attached risk depends on it.
data pipelinenot neededNot needed: there is no user interface; the integration run covers the whole flow. Becomes recommended when the pipeline ships a self-service upload screen.
machine learning model servicenot neededNot needed: the service has no user interface; the release smoke covers the endpoint. Becomes recommended when a review screen for operators ships.

Done conditions used on playbook pages

Each mandatory cell has its own done condition, written for that product type.

Tools

Alphabetical. Not recommendations.

16 tools shown

end-to-end functional tests

  • AppiumCross-platform mobile UI automation over WebDriver. iosandroidopen-source
  • ApplitoolsVisual comparison of screens across builds. webiosandroidcommercial
  • BrowserStackCloud of real devices and browsers. webiosandroidcommercial
  • CypressBrowser-based end-to-end testing with a visual runner. webfreemium
  • DeepEvalUnit-style evaluation for LLM applications. apiopen-source
  • DetoxGray-box end-to-end testing for React Native apps. iosandroidopen-source
  • EspressoAndroid UI testing framework from Google. androidopen-source
  • LambdaTestCloud browser and device grid. webiosandroidcommercial
  • MaestroMobile UI flows written in YAML. iosandroidfreemium
  • PercyVisual review of UI changes. webcommercial
  • PlaywrightBrowser automation for Chromium, Firefox and WebKit. webopen-source
  • PostmanAPI client with collections and scripted assertions. apifreemium
  • PromptfooEvaluation and regression tests for LLM prompts and outputs. apiwebopen-source
  • Sauce LabsCloud test infrastructure for web and mobile. webiosandroidcommercial
  • SeleniumWebDriver-based browser automation. webopen-source
  • XCTestApple's framework for unit and UI tests. iosopen-source