Data integrity and migration testing
Verifying data stays correct through writes, migrations, imports and exports.
Mandatory for 27 of 46 product types. Usually run by a developer.
What it catches
- lost or corrupted records
- rounding
- duplicate rows
What it misses
- UI issues
Which product types need it and why
| Product type | Status | Reason or condition |
|---|---|---|
| accounting and invoicing SaaS | mandatory | Risks: Records silently wrong, Fine or enforcement by a regulator |
| clinical trial management software | mandatory | Risks: Records silently wrong, Fine or enforcement by a regulator |
| crypto exchange | mandatory | Risks: Money leaves or fails to arrive |
| data pipeline | mandatory | Risks: Records silently wrong, Users or operators act on wrong numbers, Fine or enforcement by a regulator |
| dating app | mandatory | Risks: Personal or sensitive data exposed |
| desktop application | mandatory | Risks: Records silently wrong |
| e-signature service | mandatory | Risks: Records silently wrong |
| ecommerce store | mandatory | Risks: Money leaves or fails to arrive |
| electronic health record system | mandatory | Risks: Records silently wrong, Users or operators act on wrong numbers |
| email marketing platform | mandatory | Risks: Records silently wrong, Fine or enforcement by a regulator |
| ERP system | mandatory | Risks: Records silently wrong, Users or operators act on wrong numbers |
| EV charging app | mandatory | Risks: Money leaves or fails to arrive |
| expense management tool | mandatory | Risks: Records silently wrong |
| fleet and logistics tracking | mandatory | Risks: Records silently wrong |
| insurance app | mandatory | Risks: Records silently wrong, Money leaves or fails to arrive, Users or operators act on wrong numbers |
| lending platform | mandatory | Risks: Records silently wrong, Fine or enforcement by a regulator |
| loyalty program app | mandatory | Risks: Records silently wrong, Money leaves or fails to arrive |
| online marketplace | mandatory | Risks: Fine or enforcement by a regulator |
| machine learning model service | mandatory | Risks: Users or operators act on wrong numbers, Physical or psychological harm to a user, Fine or enforcement by a regulator |
| mobile banking app | mandatory | Risks: Records silently wrong, Money leaves or fails to arrive |
| mobile game | mandatory | Risks: Money leaves or fails to arrive |
| neobank | mandatory | Risks: Records silently wrong, Money leaves or fails to arrive |
| peer-to-peer payment app | mandatory | Risks: Money leaves or fails to arrive |
| pharmacy delivery app | mandatory | Risks: Fine or enforcement by a regulator |
| ride-hailing app | mandatory | Risks: Users or operators act on wrong numbers |
| subscription box service | mandatory | Risks: Money leaves or fails to arrive, Fine or enforcement by a regulator |
| stock trading app | mandatory | Risks: Users or operators act on wrong numbers |
| government services portal | recommended | Becomes mandatory when the portal hands data to more than one case system. |
| grocery delivery app | recommended | Becomes mandatory when shoppers are charged the wrong amount for items priced by weight or for substitutes. |
| identity and access management service | recommended | Becomes mandatory when a second regulator action on reuse of security data is attached. |
| mental health app | recommended | Becomes mandatory when clinicians review mood history to adjust treatment. |
| patient portal | recommended | Becomes mandatory when the portal moves to a new record system or migrates message history. |
| software as a medical device | recommended | Becomes mandatory when clinicians make treatment decisions from the uploaded history. |
| telemedicine app | recommended | Becomes mandatory when visit notes are exported to a partner EHR or migrated between systems. |
| event ticketing platform | recommended | Becomes mandatory when resale between fans ships. |
| travel booking app | recommended | Becomes mandatory when travellers receive confirmations with the wrong flight time or date. |
| wearable companion app | recommended | Becomes mandatory when readings are shared with a clinician or insurer. |
| buy now pay later service | not needed | Not needed as a separate test: plan balances are covered by the payment-sandbox and integration cases. Becomes recommended when plan data is migrated between ledgers or furnished to credit bureaus. |
| smart home or IoT app | not needed | Not needed: the app keeps device state, not records that are reported or billed. Becomes recommended when energy-use history is shown or exported. |
| kids app | not needed | Not needed: no financial or official record is kept. Becomes recommended when progress reports go to schools. |
| language learning app | not needed | Not needed: no record above is financial or official. Becomes recommended when the app issues certificates or level scores to schools. |
| LLM-based assistant | not needed | Not needed: the assistant stores chats as written and computes no figures. Becomes recommended when answers write back to customer records. |
| messaging app | not needed | Not needed: message history stays in one store. Becomes recommended when message history migrates to a new store. |
| SDK or client library | not needed | Not needed: the SDK stores no records of its own. Becomes recommended when it adds a local database or sync. |
| social network | not needed | Not needed: no figures or records are computed for users to act on. Becomes recommended when creator payouts are calculated from views. |
| video conferencing tool | not needed | Not needed: no data-corruption risk is attached. Becomes recommended when meeting records feed billing or compliance archives. |
Done conditions used on playbook pages
Each mandatory cell has its own done condition, written for that product type.
- accounting and invoicing SaaS (Fintech): Importing the same bank statement twice leaves one copy of each transaction, and the VAT report totals equal the sum of the posted ledger lines.
- clinical trial management software (Healthcare): After a protocol-amendment migration, a reconciliation report shows every subject, visit and value matches the pre-migration export, and audit trails are carried over unchanged.
- crypto exchange (Fintech): After each ledger close and each wallet movement, customer balances per asset in the ledger sum to no more than on-chain and custodial holdings, and no account other than approved market makers carries a negative balance.
- data pipeline (Infrastructure and developer products): For each run, row counts and control totals reconcile between source extract, staging and target, and a mismatch stops publication.
- dating app (Consumer apps): After a member deletes the account, a support lookup finds no profile, photos or messages, and matched members no longer see the chat.
- desktop application (Infrastructure and developer products): A test profile with known files keeps the same file list and contents before and after each update, including files saved in redirected folders.
- e-signature service (B2B SaaS): Editing a completed document makes signature validation fail, and editing a document mid-flow restarts signing for everyone who already signed.
- ecommerce store (Commerce): A daily report shows every cancelled or unshipped order with a card refund, not store credit, unless the shopper chose credit.
- electronic health record system (Healthcare): Medication lists and allergies for a sample of test patients match field for field between the sending and receiving records, and each required quality report produces a value.
- email marketing platform (B2B SaaS): After a list import or merge, previously suppressed contacts stay suppressed and every imported contact shows a consent source and date.
- ERP system (B2B SaaS): After migration, the trial balance, open payables and employee records match the legacy system to the cent and to the record.
- EV charging app (Infrastructure and developer products): For a sample of sessions, the kWh on each receipt matches the charger's meter values and the amount charged matches the price shown at start.
- expense management tool (Fintech): For a sample of approved expenses, each stored receipt image opens, shows a readable amount and date, and is linked to the same expense line after edits.
- fleet and logistics tracking (Infrastructure and developer products): After an edit to a driver's duty log, the original entry is still retrievable, the edit shows who made it, and the driver is asked to accept or reject it.
- insurance app (Fintech): After each rating data load, a reconciliation report shows no renewal whose premium differs from a recalculation with the filed rates.
- lending platform (Fintech): The credit bureau file for a set of test loans shows the same status, balance and days past due as the servicing screen, including a loan paid on its due date.
- loyalty program app (Commerce): After a ledger migration or program merge, every member's points balance and tier on a full export matches the old system at the published conversion ratio.
- online marketplace (Commerce): A daily ledger report shows, for every order, buyer charge equal to seller payout plus fees plus refunds, and lists every high-volume seller with verified details.
- machine learning model service (Infrastructure and developer products): On a held-out set labelled by people, accuracy and false-positive results are recorded per group and per input quality, and the release record shows them.
- mobile banking app (Fintech): After the overnight batch, balances, interest and statement lines for a seeded set of test accounts match the values worked out by hand in the reconciliation sheet.
- mobile game (Consumer apps): For a sample of sandbox purchases, every charge in the store's purchase records matches a purchase the player confirmed in the game, and each purchased item appears in the player's inventory once.
- neobank (Fintech): After each partner bank statement, the sum of customer balances in the app ledger equals the pooled account statement; a run that finds a difference lists it by customer.
- peer-to-peer payment app (Fintech): After each statement from the app's bank, the sum of user balances, pending sends and cash-outs equals the funds held there; a run that finds a difference lists it by user.
- pharmacy delivery app (Healthcare): A test controlled-substance refill past its prescription's refill limit, and one with no valid prescription on file, both stop in the pharmacist queue with a visible reason.
- ride-hailing app (Commerce): Every driver deactivation in a test week traces to a named human reviewer, and a sample of fraud flags rechecked by hand matches the recorded reason.
- subscription box service (Commerce): After each renewal run, the report shows no box shipped and no card charged for a skipped month or a cancelled or paused subscription, and no box queued for shipment on a subscription without a stored consent record.
- stock trading app (Fintech): Cash balance, buying power and position values on the app match the clearing firm's end-of-day records for a set of test accounts, including accounts with open options.
Tools
Alphabetical. Not recommendations.
data integrity and migration testing
- dbt testsTests on warehouse models inside dbt. apiopen-source
- Great ExpectationsData validation with declarative expectations. apiopen-source