Analytics and event tracking tests
Verifying that events fire with the right payload at the right moment.
Mandatory for 11 of 46 product types. Usually run by a product manager.
What it catches
- missing or duplicate events
- wrong attribution
What it misses
- product logic
Which product types need it and why
| Product type | Status | Reason or condition |
|---|---|---|
| data pipeline | mandatory | Risks: Users or operators act on wrong numbers |
| dating app | mandatory | Risks: Fine or enforcement by a regulator |
| kids app | mandatory | Risks: Fine or enforcement by a regulator, Personal or sensitive data exposed |
| language learning app | mandatory | Risks: Fine or enforcement by a regulator |
| mental health app | mandatory | Risks: Personal or sensitive data exposed, Fine or enforcement by a regulator |
| machine learning model service | mandatory | Risks: Users or operators act on wrong numbers, Physical or psychological harm to a user |
| patient portal | mandatory | Risks: Personal or sensitive data exposed, Fine or enforcement by a regulator |
| pharmacy delivery app | mandatory | Risks: Fine or enforcement by a regulator, Personal or sensitive data exposed |
| social network | mandatory | Risks: Fine or enforcement by a regulator |
| telemedicine app | mandatory | Risks: Personal or sensitive data exposed, Fine or enforcement by a regulator |
| wearable companion app | mandatory | Risks: Personal or sensitive data exposed |
| ecommerce store | recommended | Becomes mandatory when ad spend or stock decisions are made from the purchase events. |
| LLM-based assistant | recommended | Becomes mandatory when a third-party analytics SDK runs on the chat screen. |
| mobile game | recommended | Becomes mandatory when an ad or attribution SDK is added to a build rated for children. |
| buy now pay later service | optional | When merchants are billed on attributed sales, a lost conversion event misstates merchant reports. |
| electronic health record system | optional | May help confirm that quality-measure events are recorded when clinicians complete documented steps. |
| email marketing platform | optional | May help: catches open and click reports that do not match what seed inboxes did. |
| loyalty program app | optional | Optional: helps measure promotion uptake, no attached risk depends on it. |
| subscription box service | optional | May help: churn and retention numbers drive box planning. |
| accounting and invoicing SaaS | not needed | Not needed: no attached risk depends on product analytics. Becomes recommended when usage events drive billing. |
| clinical trial management software | not needed | Not needed: no product analytics decide anything in a validated trial system. Becomes recommended when a participant-facing app adds usage tracking. |
| crypto exchange | not needed | Not needed: no attached risk depends on analytics events. Becomes recommended when trading dashboards for customers rely on event data. |
| desktop application | not needed | Not needed: no attached risk depends on event accuracy. Becomes recommended when usage events drive licence counts. |
| e-signature service | not needed | Not needed: no wrong-decisions risk is attached. Becomes recommended when envelope reports drive customer billing. |
| ERP system | not needed | Not needed: product usage tracking feeds no attached risk. Becomes recommended when usage data drives licence counts. |
| EV charging app | not needed | Not needed: reliability data comes from charger status, not app analytics. Becomes recommended when app events feed published reliability figures. |
| expense management tool | not needed | Not needed: no attached risk depends on product analytics. Becomes recommended when spend dashboards drive budget decisions. |
| fleet and logistics tracking | not needed | Not needed: fleet reports come from telematics records, not app analytics. Becomes recommended when driver scores are built from app events. |
| government services portal | not needed | Not needed: no risk above depends on analytics events. Becomes recommended when third-party analytics scripts are added to signed-in pages. |
| grocery delivery app | not needed | Not needed: charges and refunds come from the order ledger, not analytics events. Becomes recommended when analytics drive slot pricing. |
| identity and access management service | not needed | Not needed: no wrong-decisions risk is attached. Becomes recommended when risk scores from sign-in analytics block users. |
| insurance app | not needed | Not needed: no attached risk depends on product analytics. Becomes recommended when analytics events feed pricing or fraud models. |
| smart home or IoT app | not needed | Not needed: no decision or claim rests on app analytics here. Becomes recommended when usage data from the home is shared with partners. |
| lending platform | not needed | Not needed: no attached risk depends on product analytics events. Becomes recommended when funnel metrics drive credit policy changes. |
| online marketplace | not needed | Not needed: no evidenced risk depends on event data. Becomes recommended when seller rankings or fees are computed from tracked events. |
| messaging app | not needed | Not needed: no ad or analytics SDK reads chat data. Becomes recommended when an ad or attribution SDK is added. |
| mobile banking app | not needed | Not needed: no attached risk depends on in-app analytics events. Becomes recommended when fraud or complaint reporting relies on app event data. |
| neobank | not needed | Not needed: no attached risk depends on product analytics events. Becomes recommended when fraud scoring uses in-app event data. |
| peer-to-peer payment app | not needed | Not needed: no attached risk depends on analytics events. Becomes recommended when fraud models use in-app event data. |
| ride-hailing app | not needed | Not needed: no attached risk depends on product analytics events. Becomes recommended when surge pricing reads demand from analytics events. |
| software as a medical device | not needed | Not needed while the app sends no product analytics. Becomes recommended when an analytics SDK is added and must be checked for health data leaving the device. |
| SDK or client library | not needed | Not needed: no wrong-decisions risk is attached. Becomes recommended when the SDK's own event counts drive customer billing. |
| event ticketing platform | not needed | Not needed: sales and seat counts come from the order ledger. Becomes recommended when analytics events drive dynamic pricing. |
| stock trading app | not needed | Not needed: no attached risk depends on product analytics. Becomes recommended when in-app prompts are targeted using trading events. |
| travel booking app | not needed | Not needed: bookings and refunds come from the booking ledger. Becomes recommended when analytics events drive ranking or pricing. |
| video conferencing tool | not needed | Not needed: no wrong-decisions risk is attached. Becomes recommended when usage reports drive seat billing. |
Done conditions used on playbook pages
Each mandatory cell has its own done condition, written for that product type.
- data pipeline (Infrastructure and developer products): Each job emits start, end, rows read and rows written, and a run with no end event or a write count of zero raises an alert.
- dating app (Consumer apps): A network capture from first launch to first match shows no GPS coordinates, gender preference or orientation field sent to any ad or analytics domain before consent.
- kids app (Education): A network capture from install through first play shows no advertising ID, location or device serial sent to any third-party domain.
- language learning app (Education): A network capture during a lesson on a child profile shows no advertising identifier and no voice clip sent to any advertising or analytics domain.
- mental health app (Healthcare): A network capture of onboarding, mood logging and session booking shows no questionnaire answer, mood value or condition name sent to any advertising or analytics domain.
- machine learning model service (Infrastructure and developer products): Production logs record each score, threshold and the action taken, and a weekly review compares a reviewed sample against the model output.
- patient portal (Healthcare): A network capture of sign-in, results, messages and appointment pages shows no request to an advertising or analytics domain carrying a name, result, page path or signed-in flag.
- pharmacy delivery app (Healthcare): A network capture of browsing, adding and ordering a medication shows no drug name, condition or prescription detail sent to any advertising or analytics endpoint.
- social network (Consumer apps): A network capture shows phone numbers and emails collected for sign-in security never sent to advertising or analytics endpoints.
- telemedicine app (Healthcare): A network capture of sign-up, symptom intake, booking and visit screens shows no symptom, drug name or visit reason sent to any advertising or analytics domain.
- wearable companion app (Healthcare): A network capture during pairing, sync and browsing shows no heart rate, sleep or cycle data sent to an analytics or advertising endpoint the user did not authorise.
Tools
Alphabetical. Not recommendations.
analytics and event tracking tests
- AvoTracking plan and event validation. webiosandroidfreemium
- Segment ProtocolsTracking plan validation for Segment events. webiosandroidcommercial