How to test an identity and access management service
Direct answer
Shipped untested, an identity and access management service accepts a wrong password through a cached-credential shortcut in delegated directory sign-in on apps without MFA, and lets the phone numbers and emails collected for MFA enrolment and recovery end up in a marketing or analytics export. 10 tests are mandatory here, each tied to a risk in the map below. After the release smoke check, start the risk tests with regression tests: a wrong password through the directory connector, a session revoked in the admin console and recovery without the enrolled MFA factor must all fail again on every change to authentication code.
10 mandatory testsfirst: regression suite
What counts as an identity and access management service
An identity and access management service here provides SSO, MFA, a user directory and permissions to other apps through hosted sign-in pages, an admin console and an API. It counts when relying apps sign their users in through it, when it stores MFA and recovery contact data, and when its support staff handle customer browser traces. A single app's own login screen is out of scope, and so is a password manager that stores credentials for the user.
Adjacent product types not covered here
- e-signature service: An e-signature service proves who signed one document and that it did not change, while an identity service decides every sign-in for every relying app of every tenant.
- ERP system: An ERP system exposes files from its environment through an internet-facing component left unpatched, while an identity service exposes a customer tenant through a session token left in a support upload.
- video conferencing tool: A video conferencing tool can be joined from a link with no account, so meeting intruders lead instead of skipped password checks and token replay.
Risk map
| Risk | Grade | Note | Evidence | |
|---|---|---|---|---|
| Fraud or abuse by users fraud-abuse | money | primary | A cached-credential shortcut in delegated directory sign-in accepts a wrong password for a user who signed in before, on apps where MFA is not enforced. | |
| Accessibility complaint or lawsuit accessibility-liability | legal | primary | The sign-in, MFA and recovery screens make users recall a password or solve a puzzle, with paste, password managers and passkeys blocked. | |
| Fine or enforcement by a regulator regulatory-penalty | legal | primary | Phone numbers and email addresses collected for MFA enrolment and account recovery end up in a marketing, ads or analytics export. | |
| Personal or sensitive data exposed data-breach | data | primary | A customer admin's HAR file on a support ticket carries a live session token issued by the identity service, so whoever reads the ticket acts as that admin in a tenant whose sign-ins the customer's other apps trust, or a compromised workstation of an outsourced support engineer exposes the support tools that act on customer tenants. | |
| Third-party dependency fails dependency-failure | outage | practitioner single incident, no second source | A quota or database layer under the token-validation service fails, and SSO sign-in errors at once in every relying app of every tenant. |
Mandatory tests (10)
Step 1: release smoke tests
Done when: On every build, a test user signs in through SAML and OIDC to a sample app, a wrong password is refused, and sign-out ends the session in the sample app.
Tool class: browser automation. Who usually runs it: QA engineer.
Step 2: regression suite
Done when: On every release, a wrong password through the directory connector is refused, a session revoked in the admin console no longer opens the sample relying app, recovery still asks for the user's enrolled MFA factor, and any failure blocks the release.
Tool class: test management. Who usually runs it: QA engineer.
Step 3: penetration testing
Done when: A tester holding a HAR file or copied session cookie from a support ticket cannot reuse it to reach the admin console, and replay from a new device and IP is refused.
Tool class: proxy. Who usually runs it: security specialist.
Step 4: unit tests
Why:fraud-abuse
Done when: Credential and cache-key checks refuse a wrong password for usernames of every length in the fixed sample set, including the longest the directory allows.
Tool class: unit framework. Who usually runs it: developer.
Step 5: integration tests
Why:fraud-abuse
Done when: With a directory connector enabled and MFA off for a test app, a sign-in with the right username and a wrong password is refused straight after that user's successful sign-in.
Tool class: API client. Who usually runs it: developer.
Step 6: end-to-end functional tests
Why:fraud-abuse
Done when: Enrolment, step-up MFA, password reset and account recovery each end in the expected state, and recovery never skips the user's enrolled factor.
Tool class: browser automation. Who usually runs it: QA engineer.
Step 7: compliance checks
Why:regulatory-penaltydata-breach
Done when: MFA phone numbers and recovery emails are tagged security-only, no marketing or analytics export includes them, and support uploads are scrubbed of tokens on receipt.
Tool class: test management. Who usually runs it: product manager.
Step 8: accessibility testing
Done when: A screen-reader user and a keyboard-only user complete sign-in, MFA enrolment and recovery with a password manager or passkey and without solving a puzzle.
Tool class: accessibility checker. Who usually runs it: accessibility specialist.
Step 9: security scanning
Why:data-breach
Done when: The scan of the admin console, token endpoints and support upload flow has no open high findings.
Tool class: DAST scanner. Who usually runs it: security specialist.
Step 10: resilience and failure testing
Done when: With the session store or quota service behind token validation cut in staging, existing sessions stay valid until expiry, new sign-ins show a clear error, and service returns without a manual restart.
Tool class: chaos tool. Who usually runs it: developer.
Recommended and optional tests
contract testsrecommended
Becomes mandatory when a claim change breaks relying apps in a recorded incident.
performance and load testingrecommended
Becomes mandatory when outage-under-load evidence for sign-in peaks is attached.
data integrity and migration testingrecommended
Becomes mandatory when a second regulator action on reuse of security data is attached.
exploratory testingoptional
Optional: finds privilege paths that scripted cases miss.
usability testingoptional
Optional: helps when recovery calls to support rise.
device and browser compatibilityoptional
Optional: helps when relying apps embed sign-in in mobile webviews.
Not needed and why
Not needed: assumes hosted pages in one language. Becomes recommended when tenants enable a second locale.
analytics and event tracking tests
Not needed: no wrong-decisions risk is attached. Becomes recommended when risk scores from sign-in analytics block users.
Order of work
First release
- release smoke tests: Sign a test user in through SAML and OIDC to a sample app, confirm a wrong password is refused, and confirm sign-out ends the session in the sample app.
- regression suite: Rerun a wrong password through the directory connector, a session revoked in the admin console against the sample relying app and account recovery against the enrolled MFA factor, and block the release on any failure.
- penetration testing: Have a tester reuse a HAR file or copied session cookie from a support ticket to reach the admin console, including a replay from a new device and IP.
- unit tests: Run the credential and cache-key checks against a wrong password for usernames of every length in the fixed sample set, up to the longest the directory allows.
- integration tests: Enable a directory connector with MFA off for a test app, sign a user in, then try the right username with a wrong password straight after.
- end-to-end functional tests: Run enrolment, step-up MFA, password reset and account recovery to their expected states and confirm recovery never skips the user's enrolled factor.
- compliance checks: Confirm MFA phone numbers and recovery emails are tagged security-only, appear in no marketing or analytics export, and that support uploads are scrubbed of tokens on receipt.
- accessibility testing: Have a screen-reader user and a keyboard-only user complete sign-in, MFA enrolment and recovery with a password manager or passkey and without solving a puzzle.
- security scanning: Scan the admin console, token endpoints and support upload flow and clear every open high finding.
- resilience and failure testing: Cut the session store or quota service behind token validation in staging and confirm existing sessions stay valid, new sign-ins show a clear error, and service returns without a manual restart.
Ongoing cadence
- release smoke tests: On every build.
- regression suite: On every release that changes authentication code.
- unit tests: After each change to credential hashing or cache keys.
- integration tests: When a directory connector or delegated authentication mode is added or changed.
- compliance checks: When a new marketing, analytics or CRM export is connected, and after each change to the support upload flow.
- resilience and failure testing: After each change to the session store, quota service or other shared service behind token validation.
- contract tests: When token claims or SAML attributes change.
Tools
Listed alphabetically inside each test type, filtered to the tests above and to the platforms of identity and access management service (web, api). Not recommendations.
release smoke tests
regression suite
- CypressBrowser-based end-to-end testing with a visual runner. webfreemium
- DeepEvalUnit-style evaluation for LLM applications. apiopen-source
- PlaywrightBrowser automation for Chromium, Firefox and WebKit. webopen-source
- PromptfooEvaluation and regression tests for LLM prompts and outputs. apiwebopen-source
- TestRailTest case and run management. webiosandroidapicommercial
- XrayTest management inside Jira. webiosandroidapicommercial
penetration testing
- Burp SuiteProxy and scanner for web application security testing. webapifreemium
unit tests
integration tests
- Charles ProxyHTTP proxy for inspecting and throttling traffic. iosandroidwebcommercial
- JestJavaScript test runner with mocking and snapshots. webapiopen-source
- JUnitTest framework for the JVM. apiandroidopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- pytestTest framework for Python with fixtures and plugins. apiopen-source
- VitestTest runner for Vite-based JavaScript and TypeScript projects. webapiopen-source
end-to-end functional tests
- ApplitoolsVisual comparison of screens across builds. webiosandroidcommercial
- BrowserStackCloud of real devices and browsers. webiosandroidcommercial
- CypressBrowser-based end-to-end testing with a visual runner. webfreemium
- DeepEvalUnit-style evaluation for LLM applications. apiopen-source
- LambdaTestCloud browser and device grid. webiosandroidcommercial
- PercyVisual review of UI changes. webcommercial
- PlaywrightBrowser automation for Chromium, Firefox and WebKit. webopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- PromptfooEvaluation and regression tests for LLM prompts and outputs. apiwebopen-source
- Sauce LabsCloud test infrastructure for web and mobile. webiosandroidcommercial
- SeleniumWebDriver-based browser automation. webopen-source
accessibility testing
- axe-coreAccessibility rules engine for automated checks. webopen-source
- LighthouseAudits for accessibility and performance in Chrome. webopen-source
- Pa11yCommand-line accessibility testing. webopen-source
security scanning
- Burp SuiteProxy and scanner for web application security testing. webapifreemium
- GarakProbing LLM applications for prompt injection and leakage. apiopen-source
- mitmproxyInteractive HTTPS proxy. iosandroidwebapiopen-source
- OWASP ZAPDynamic application security scanner. webapiopen-source
- SchemathesisProperty-based testing from OpenAPI and GraphQL schemas. apiopen-source
- SemgrepStatic analysis with custom rules. apiwebiosandroidfreemium
- SnykDependency and code vulnerability scanning. apiwebiosandroidfreemium
- TrivyScanner for containers, dependencies and configuration. apiopen-source
resilience and failure testing
- Chaos MeshFault injection for Kubernetes. apiopen-source
- GremlinManaged fault injection. apicommercial
- ToxiproxyTCP proxy for simulating network failures. apiopen-source
contract tests
- PactConsumer-driven contract testing with a broker. apiopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- SchemathesisProperty-based testing from OpenAPI and GraphQL schemas. apiopen-source
performance and load testing
- ArtilleryLoad testing with YAML scenarios. apiwebfreemium
- GatlingLoad testing with Scala, Java or Kotlin scenarios. apiwebfreemium
- JMeterLoad testing tool from the Apache Foundation. apiwebopen-source
- k6Load testing scripted in JavaScript. apiwebopen-source
- LighthouseAudits for accessibility and performance in Chrome. webopen-source
- LocustLoad testing scripted in Python. apiwebopen-source
data integrity and migration testing
- dbt testsTests on warehouse models inside dbt. apiopen-source
- Great ExpectationsData validation with declarative expectations. apiopen-source
What to send to the person who will test
Send the checklist and the brief below. The printable checklist for identity and access management service has the same 10 items with checkboxes.
Download the checklistMarkdownCSVBrief (Markdown)
Acceptance criteria wording
A release of the identity and access management service is accepted when a test user signs in through SAML and OIDC to a sample app, a wrong password is refused, including straight after a successful sign-in through a directory connector with MFA off, and sign-out ends the session. A session cookie or HAR file copied from a support ticket cannot reach the admin console, and no marketing or analytics export contains MFA phone numbers or recovery emails.
Brief
# Test brief: identity and access management service Product type: identity and access management service. SSO, MFA, user directory and permissions for other apps. Risk map and evidence: https://howtotestaproduct.com/how-to-test/identity-provider/ Printable checklist: https://howtotestaproduct.com/checklists/identity-provider/ Run these 10 mandatory tests in this order. A test is done only when its done condition is true on the build you will release. 1. release smoke tests (risks: fraud-abuse, data-breach). Tool class: browser automation. Usually run by: QA engineer. Done when: On every build, a test user signs in through SAML and OIDC to a sample app, a wrong password is refused, and sign-out ends the session in the sample app. 2. regression suite (risks: fraud-abuse, data-breach). Tool class: test management. Usually run by: QA engineer. Done when: On every release, a wrong password through the directory connector is refused, a session revoked in the admin console no longer opens the sample relying app, recovery still asks for the user's enrolled MFA factor, and any failure blocks the release. 3. penetration testing (risks: data-breach, fraud-abuse). Tool class: proxy. Usually run by: security specialist. Done when: A tester holding a HAR file or copied session cookie from a support ticket cannot reuse it to reach the admin console, and replay from a new device and IP is refused. 4. unit tests (risks: fraud-abuse). Tool class: unit framework. Usually run by: developer. Done when: Credential and cache-key checks refuse a wrong password for usernames of every length in the fixed sample set, including the longest the directory allows. 5. integration tests (risks: fraud-abuse). Tool class: API client. Usually run by: developer. Done when: With a directory connector enabled and MFA off for a test app, a sign-in with the right username and a wrong password is refused straight after that user's successful sign-in. 6. end-to-end functional tests (risks: fraud-abuse). Tool class: browser automation. Usually run by: QA engineer. Done when: Enrolment, step-up MFA, password reset and account recovery each end in the expected state, and recovery never skips the user's enrolled factor. 7. compliance checks (risks: regulatory-penalty, data-breach). Tool class: test management. Usually run by: product manager. Done when: MFA phone numbers and recovery emails are tagged security-only, no marketing or analytics export includes them, and support uploads are scrubbed of tokens on receipt. 8. accessibility testing (risks: accessibility-liability). Tool class: accessibility checker. Usually run by: accessibility specialist. Done when: A screen-reader user and a keyboard-only user complete sign-in, MFA enrolment and recovery with a password manager or passkey and without solving a puzzle. 9. security scanning (risks: data-breach). Tool class: DAST scanner. Usually run by: security specialist. Done when: The scan of the admin console, token endpoints and support upload flow has no open high findings. 10. resilience and failure testing (risks: dependency-failure). Tool class: chaos tool. Usually run by: developer. Done when: With the session store or quota service behind token validation cut in staging, existing sessions stay valid until expiry, new sign-ins show a clear error, and service returns without a manual restart. Acceptance criteria wording: A release of the identity and access management service is accepted when a test user signs in through SAML and OIDC to a sample app, a wrong password is refused, including straight after a successful sign-in through a directory connector with MFA off, and sign-out ends the session. A session cookie or HAR file copied from a support ticket cannot reach the admin console, and no marketing or analytics export contains MFA phone numbers or recovery emails. Report back for each test: pass or fail against its done condition, the build or version tested, and the proof (screenshot, log line or report file). Licence: CC BY 4.0.
For founders: what to forward and what to ask back. For product managers: turning this page into tickets.
Questions about testing an identity and access management service
What testing does an identity and access management service need?
10 tests are mandatory, starting with a SAML and OIDC sign-in smoke run on every build and regression cases for directory-connector passwords, sessions revoked in the admin console and recovery that keeps the enrolled MFA factor. Each is tied to one of the 5 risks in the map: accepted wrong passwords, session tokens leaked through support tickets, MFA contact data reused for marketing, a shared service outage that stops all sign-ins, and sign-in screens that block password managers and passkeys. Contract, performance and data integrity tests are recommended, each with the event that makes it mandatory.
What is the minimum testing for an identity and access management service at MVP stage?
The minimum is the 10 mandatory tests, because relying apps sign users in through the service, it stores MFA and recovery contact data and its admin console is reachable from the internet from the first release. Exploratory, usability and compatibility testing are optional, and localization and analytics tests that vendors often add are not needed until the conditions on the page change.
Which tests in a vendor test plan for an identity and access management service are not needed?
Localization testing is not needed while hosted pages ship in one language, until tenants enable a second locale. Analytics tests are not needed while no wrong-decisions risk is attached, until risk scores from sign-in analytics block users.
What should I send to the person who will test an identity and access management service?
Send a sample app connected through SAML and OIDC, a test tenant with a directory connector and MFA off for one app, and the fixed username set up to the longest length the directory allows. Add the list of marketing, analytics and CRM exports for the compliance check and staging access to the session store and quota service for the resilience test.
When is test automation worth it for an identity and access management service?
Automation pays for the sign-in smoke run on every build, the regression cases for directory-connector passwords, revoked sessions in the sample relying app and MFA recovery, and the cache-key unit tests across username lengths. The support-ticket token replay is run by a security specialist, and the accessibility run uses a real screen-reader user.
Evidence
ev-0037standardprimary
NIST SP 800-63B: Digital Identity Guidelines, Authentication and Authenticator Management, National Institute of Standards and Technology
SP 800-63B sets requirements for authenticators, multi-factor authentication, session management and throttling of guessing attempts.
published undated · verified 2026-10-07
ev-0250post-mortemprimary
Okta AD/LDAP Delegated Authentication - Username Above 52 Characters Security Advisory, Okta
If the vulnerability was exploited, this could allow a user to authenticate with the stored cache key of a previous successful authentication.
Official security advisory: a cache key built with a hash that truncates long input let sign-ins through delegated directory authentication succeed without checking the password when MFA was not applied. Supports fraud-abuse for identity providers that cache directory credentials.
published 2024-11-01 · verified 2026-10-07
ev-0016standardprimary
Web Content Accessibility Guidelines (WCAG) 2.2, W3C
WCAG 2.2 is the W3C standard that accessibility law and procurement rules reference; level AA is the usual target.
published undated · verified 2026-10-07
ev-0251regulatorprimary
FTC Charges Twitter with Deceptively Using Account Security Data to Sell Targeted Ads, Federal Trade Commission
Twitter asked users to give their phone numbers and email addresses to protect their accounts.
Regulator action: phone numbers and emails collected for two-factor authentication and account recovery were used for ad targeting, breaching an earlier order. Transfers to identity providers, which collect the same MFA and recovery data for their customers' users.
published 2022-05-25 · verified 2026-10-07
ev-0244post-mortemprimary
Unauthorized Access to Okta's Support Case Management System: Root Cause and Remediation, Okta
Some of these files were HAR files that contained session tokens which could in turn be used for session hijacking attacks.
Official post-mortem: an attacker read files customers had attached to cases in the company's support case system, and browser HAR files in those cases held live session tokens. Supports data-breach for helpdesk software through ticket attachments.
published 2023-11-03 · verified 2026-10-07
ev-0245post-mortemprimary
Tracking Unauthorized Access to Okta's Support System, Okta
Okta recommends sanitizing all credentials and cookies/session tokens within a HAR file before sharing it.
Official incident notice: files uploaded to support cases were accessed, and the company tells customers to strip cookies and tokens from HAR files before attaching them. Supports data-breach for helpdesk software that accepts diagnostic file uploads.
published 2023-10-20 · verified 2026-10-07
ev-0249post-mortemprimary
Okta's Investigation of the January 2022 Compromise, Okta
Official investigation update: the workstation of a support engineer at a contracted sub-processor was compromised, and screenshots of internal support tools were published. Supports data-breach for identity providers whose outsourced support staff can act on customer tenants.
published 2022-03-24 · verified 2026-10-07
ev-0252post-mortemprimary
Google Cloud Infrastructure Components incident (User ID Service authentication outage), Google Cloud
The Google User ID Service maintains a unique identifier for every account and handles authentication credentials for OAuth tokens and cookies.
Official incident report: an automated quota system cut the quota of the account database behind the authentication service after a migration reported zero usage, and authentication lookups failed across dependent services. Supports dependency-failure for identity services built on internal platform layers.
published undated · verified 2026-10-07
Disclaimer
This page lists tests and risks for a product category, not for your product. Risks are attached only with the evidence shown in the Evidence section. Tools are listed alphabetically and are not recommendations. Nothing here is legal, regulatory or security advice; where a regulator or a store sets a requirement, their page is linked and takes precedence.