Integration tests
Automated checks of two or more components together, often with a real database or queue.
Mandatory for 23 of 46 product types. Usually run by a developer.
What it catches
- wrong wiring
- schema drift
- transaction faults
What it misses
- user-visible flows
- third-party outages
Which product types need it and why
| Product type | Status | Reason or condition |
|---|---|---|
| accounting and invoicing SaaS | mandatory | Risks: Third-party dependency fails, Records silently wrong |
| buy now pay later service | mandatory | Risks: Money leaves or fails to arrive |
| clinical trial management software | mandatory | Risks: Records silently wrong |
| data pipeline | mandatory | Risks: Records silently wrong, Third-party dependency fails |
| ecommerce store | mandatory | Risks: Money leaves or fails to arrive, Fine or enforcement by a regulator |
| electronic health record system | mandatory | Risks: Physical or psychological harm to a user, Records silently wrong |
| email marketing platform | mandatory | Risks: App rejected or removed by a store, Records silently wrong |
| ERP system | mandatory | Risks: Records silently wrong |
| EV charging app | mandatory | Risks: Third-party dependency fails |
| expense management tool | mandatory | Risks: Money leaves or fails to arrive, Records silently wrong |
| identity and access management service | mandatory | Risks: Fraud or abuse by users |
| insurance app | mandatory | Risks: Records silently wrong |
| lending platform | mandatory | Risks: Money leaves or fails to arrive, Records silently wrong |
| loyalty program app | mandatory | Risks: Records silently wrong |
| messaging app | mandatory | Risks: Third-party dependency fails |
| mobile banking app | mandatory | Risks: Money leaves or fails to arrive, Third-party dependency fails |
| neobank | mandatory | Risks: Third-party dependency fails, Records silently wrong |
| patient portal | mandatory | Risks: User actions lost on bad networks or devices |
| software as a medical device | mandatory | Risks: Physical or psychological harm to a user, User actions lost on bad networks or devices |
| SDK or client library | mandatory | Risks: Fine or enforcement by a regulator, Personal or sensitive data exposed |
| subscription box service | mandatory | Risks: Money leaves or fails to arrive |
| travel booking app | mandatory | Risks: Money leaves or fails to arrive |
| video conferencing tool | mandatory | Risks: Personal or sensitive data exposed |
| crypto exchange | recommended | Becomes mandatory when a new chain or token is listed. |
| dating app | recommended | Becomes mandatory when photo moderation or identity checks move to a new provider. |
| e-signature service | recommended | Becomes mandatory when the risk "Fraud or abuse by users" is attached to this product type with verified evidence (see the methodology). |
| fleet and logistics tracking | recommended | Becomes mandatory when proof of delivery is pushed into customers' own systems. |
| government services portal | recommended | Becomes mandatory when a new backend is connected. |
| grocery delivery app | recommended | Becomes mandatory when store stock comes from a partner retailer's system. |
| smart home or IoT app | recommended | Becomes mandatory when devices are sold as working with a voice assistant or another home platform. |
| LLM-based assistant | recommended | Becomes mandatory when the assistant can call tools that write data or send messages. |
| online marketplace | recommended | Becomes mandatory when a payout released before a dispute closes is recorded for this product type with verified evidence (see the methodology). |
| peer-to-peer payment app | recommended | Becomes mandatory when cash-out runs through a new bank or card network partner. |
| pharmacy delivery app | recommended | Becomes mandatory when the risk "Third-party dependency fails" or "Money leaves or fails to arrive" is attached to this product type with verified evidence (see the methodology). |
| ride-hailing app | recommended | Becomes mandatory when fares or earnings pass between separately deployed services. |
| telemedicine app | recommended | Becomes mandatory when prescriptions go to pharmacies through an e-prescribing network rather than by fax or phone. |
| event ticketing platform | recommended | Becomes mandatory when venues use their own access control system. |
| stock trading app | recommended | Becomes mandatory when the broker changes clearing firm or order router. |
| wearable companion app | recommended | Becomes mandatory when other apps read the band's data through the phone's health store. |
| machine learning model service | optional | Optional: catches wiring faults between feature store and model. |
| desktop application | not needed | Not needed: no attached risk comes from a third-party service. Becomes recommended when the app syncs with an outside service. |
| kids app | not needed | Not needed: the app has no third-party service beyond the store. Becomes recommended when a classroom or cloud save service is added. |
| language learning app | not needed | Not needed as a separate item: the store and speech services are covered by the payment and compatibility tests. Becomes recommended when a second speech provider is added. |
| mental health app | not needed | Not needed while the app has no partner systems beyond its own backend. Becomes recommended when sessions are booked into a provider's scheduling system. |
| mobile game | not needed | Not needed beyond the payment and load tests above. Becomes recommended when the game adds a third-party login or cross-platform account service. |
| social network | not needed | Not needed: no third-party app gets user data through an API. Becomes recommended when an API for third-party apps ships. |
Done conditions used on playbook pages
Each mandatory cell has its own done condition, written for that product type.
- accounting and invoicing SaaS (Fintech): A transaction posted in the sandbox bank appears once in the bank feed with the same date, amount and payee.
- buy now pay later service (Fintech): A refund, a partial capture and a cancelled order sent through the merchant API each change the shopper's plan in the app to the matching amount.
- clinical trial management software (Healthcare): Lab, eCOA and device files imported into the EDC land on the right subject and visit, with the same values and units as the source file.
- data pipeline (Infrastructure and developer products): An end-to-end run against staging copies of each source and target loads a file above the largest size seen in production with no rows lost, and when the source stream stops sending mid-run, the run is reported as stalled instead of complete.
- ecommerce store (Commerce): Order, payment and fulfilment status match across the storefront, payment provider and warehouse system for paid, shipped, cancelled and refunded orders.
- electronic health record system (Healthcare): Every order type placed in a test chart reaches its target lab, pharmacy or specialty worklist, and an order with no valid target is refused at entry with a message.
- email marketing platform (B2B SaaS): Clicking one-click unsubscribe in a Gmail or Yahoo seed inbox removes the contact once, and a link scanner opening the same link leaves the contact shown as subscribed in the audience list.
- ERP system (B2B SaaS): Every field sent by each feeder system arrives in the ledger posting, and a missing required field rejects the batch with a visible error.
- EV charging app (Infrastructure and developer products): Start, stop and meter messages between the backend and a test charger, and through a roaming partner, produce the same session record on both sides.
- expense management tool (Fintech): A card transaction from the issuer processor appears once on the cardholder's expense list with the same amount, currency and merchant, and a reversal removes it.
- identity and access management service (B2B SaaS): With a directory connector enabled and MFA off for a test app, a sign-in with the right username and a wrong password is refused straight after that user's successful sign-in.
- insurance app (Fintech): An insurance score or vehicle record returned by the data vendor lands in the policy record with the same value and tier the vendor sent.
- lending platform (Fintech): A repayment posted by the processor updates the loan balance and next due date on the servicing screen, and a returned payment reverses both.
- loyalty program app (Commerce): After points move to a new ledger or a second program merges in, the balance and tier a member sees in the app, at the point of sale and in a partner channel equal the converted balance in the new ledger, and a new purchase in each channel credits points once.
- messaging app (Consumer apps): With the cloud network gateway in front of the chat backend switched off in staging and then restored, phones reconnect without the app freezing, every message sent during the outage arrives once and each recipient gets one notification.
- mobile banking app (Fintech): A test debit card authorisation and its later settlement create one ledger entry, and no overdraft fee appears when the balance covered the amount at authorisation.
- neobank (Fintech): A card payment, a refund and an incoming transfer made in the sandbox each appear once in both the app ledger and the partner bank feed.
- patient portal (Healthcare): Every message type sent from the portal appears in the EHR inbox for the right care team and is filed to the patient's chart.
- software as a medical device (Healthcare): Each command the app sends reaches the test device once, the device confirms it, and the app shows the confirmed value rather than the requested one.
- SDK or client library (Infrastructure and developer products): A proxy capture of a sample host app shows the SDK sends only the data types in its privacy manifest, and nothing before the host app's consent call.
- subscription box service (Commerce): Subscription status, next billing date and next shipment date match across the storefront, billing provider and fulfilment system after a skip, pause and cancel.
- travel booking app (Commerce): A cancellation or schedule change sent by each connected supplier appears on the traveller's booking and opens a refund case.
- video conferencing tool (B2B SaaS): A proxy capture of app open shows no third-party SDK sending device data before the user picks that SDK's feature.
Tools
Alphabetical. Not recommendations.
integration tests
- Charles ProxyHTTP proxy for inspecting and throttling traffic. iosandroidwebcommercial
- JestJavaScript test runner with mocking and snapshots. webapiopen-source
- JUnitTest framework for the JVM. apiandroidopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- pytestTest framework for Python with fixtures and plugins. apiopen-source
- VitestTest runner for Vite-based JavaScript and TypeScript projects. webapiopen-source
- XCTestApple's framework for unit and UI tests. iosopen-source