How to test an SDK or client library
Direct answer
Shipped untested, an SDK or client library hands integrating apps sample privacy text that does not say who receives the location data it collects, and crashes host apps at launch inside the SDK when a server response changes. 9 tests are mandatory here, each tied to a risk in the map below. After the release smoke check, start the risk tests with integration tests: a network capture of a host app shows whether the SDK sends only the data types in its privacy manifest and nothing before the host app's consent call.
9 mandatory testsfirst: integration tests
What counts as an SDK or client library
An SDK or client library here is code shipped into other developers' codebases for API, iOS, Android and web hosts, installed through a package manager and updated by the host app's team. It counts when the SDK reads configuration from its own servers, collects any user or device data, or pulls third-party dependencies into host builds. A public API called over HTTP with no shipped library is out of scope, and so is an internal module used only inside one app.
Adjacent product types not covered here
- desktop application: A desktop application installs on the user's machine and its own updater can delete user folders, while an SDK runs inside someone else's app and breaks their build or launch.
- data pipeline: A data pipeline moves records between systems on a schedule, so dropped rows and reconciliation lead instead of host app crashes and privacy manifests.
- machine learning model service: A machine learning model service returns scores from one endpoint it controls, while an SDK version stays in host apps long after a fix is released.
Risk map
| Risk | Grade | Note | Evidence | |
|---|---|---|---|---|
| Fine or enforcement by a regulator regulatory-penalty | legal | primary | The sample privacy text handed to integrating apps does not say who receives the location data the SDK collects, and the regulator holds the SDK owner liable. | |
| Personal or sensitive data exposed data-breach | data | primary | A new maintainer of a transitive dependency publishes a release carrying malicious code, and host apps that update the SDK pull it into their builds. | |
| Third-party dependency fails dependency-failure | outage | primary | A change in a server response crashes host apps at launch inside the SDK, or an unpublished transitive package makes host installs and builds fail. | |
| App rejected or removed by a store store-rejection | distribution | primary | An SDK on Apple's list of commonly used SDKs ships without a privacy manifest or signature, so App Store Connect rejects host app submissions and updates that add it. |
Mandatory tests (9)
Step 1: release smoke tests
Why:dependency-failurestore-rejection
Done when: Each release candidate carries its privacy manifest and a signature that Xcode validates, installs into sample host apps for every supported platform, builds from a clean cache, launches and completes the quick-start call.
Tool class: mobile automation. Who usually runs it: QA engineer.
Step 2: integration tests
Why:regulatory-penaltydata-breach
Done when: A proxy capture of a sample host app shows the SDK sends only the data types in its privacy manifest, and nothing before the host app's consent call.
Tool class: proxy. Who usually runs it: developer.
Step 3: compliance checks
Why:regulatory-penaltystore-rejection
Done when: The privacy manifest and the sample disclosure text list every data type the SDK collects and every recipient, signed off by legal, and the binary is signed.
Tool class: test management. Who usually runs it: product manager.
Step 4: security scanning
Why:data-breach
Done when: The dependency report for the release lists no known-malicious or high-severity package, every transitive dependency is pinned in the lockfile, and maintainer changes are reviewed.
Tool class: dependency scanner. Who usually runs it: security specialist.
Step 5: device and browser compatibility
Why:store-rejectiondependency-failure
Done when: The SDK builds and runs in sample host apps on the oldest and newest supported OS, language and package manager versions, and each iOS sample host app build passes App Store Connect validation with the SDK's privacy manifest.
Tool class: device cloud. Who usually runs it: QA engineer.
Step 6: unit tests
Done when: In the test report, every server-sent config and response field in the fixed sample set passes with a null, wrong-type and missing value and no crash.
Tool class: unit framework. Who usually runs it: developer.
Step 7: contract tests
Done when: Before any server config change goes live, staging responses match the schema that every supported SDK version expects.
Tool class: contract tool. Who usually runs it: developer.
Step 8: regression suite
Done when: The public API surface diff and the crash cases rerun on every release, and a removed or changed public method without a major version bump blocks the release.
Tool class: test management. Who usually runs it: QA engineer.
Step 9: resilience and failure testing
Done when: With the SDK backend returning errors, timeouts or malformed config, the sample host app keeps running and its own features work.
Tool class: chaos tool. Who usually runs it: developer.
Recommended and optional tests
performance and load testingrecommended
Becomes mandatory when a host app reports launch timeouts caused by the SDK.
penetration testingrecommended
Becomes mandatory when the SDK handles credentials or payment data.
offline and poor-network testingrecommended
Becomes mandatory when lost-actions evidence for queued SDK calls is attached.
end-to-end functional testsoptional
Optional: catches doc drift, no attached risk depends on it.
exploratory testingoptional
Optional: finds crash paths that scripted cases miss.
Not needed and why
Not needed: the SDK has no end-user interface. Becomes recommended when it ships UI components.
Not needed: the SDK draws no screens. Becomes recommended when it ships UI components used in host apps.
Not needed: the SDK shows no user-facing text. Becomes recommended when it ships UI strings.
Order of work
First release
- release smoke tests: Check the release candidate carries its privacy manifest and a signature that Xcode validates, then install it into sample host apps for every supported platform, build from a clean cache, launch and complete the quick-start call.
- integration tests: Capture the network traffic of a sample host app and confirm the SDK sends only the data types in its privacy manifest and nothing before the host app's consent call.
- compliance checks: Have legal sign off the privacy manifest and sample disclosure text against every data type the SDK collects and every recipient, and confirm the binary is signed.
- security scanning: Run the dependency report for the release, clear every known-malicious or high-severity package, pin every transitive dependency in the lockfile and review maintainer changes.
- device and browser compatibility: Build and run the SDK in sample host apps on the oldest and newest supported OS, language and package manager versions, and pass each iOS sample host app build through App Store Connect validation.
- unit tests: Run the unit suite on every server-sent config and response field with null, wrong-type and missing values, and confirm in the test report that each passes with no crash.
- contract tests: Check staging responses against the schema every supported SDK version expects before any server config change goes live.
- regression suite: Rerun the public API surface diff and crash cases, and block the release on a removed or changed public method without a major version bump.
- resilience and failure testing: Make the SDK backend return errors, timeouts and malformed config and confirm the sample host app keeps running with its own features working.
Ongoing cadence
- release smoke tests: On every release candidate.
- contract tests: Before any server config change goes live.
- security scanning: On every dependency update and when a maintainer of a transitive dependency changes.
- compliance checks: When the SDK starts collecting a new data type or sending to a new recipient.
- device and browser compatibility: When a new OS, language or package manager version is published.
- regression suite: On every release.
- performance and load testing: When a host app reports a launch timeout caused by the SDK.
Tools
Listed alphabetically inside each test type, filtered to the tests above and to the platforms of SDK or client library (api, ios, android, web). Not recommendations.
release smoke tests
- Firebase Test LabRuns app tests on hosted devices. iosandroidfreemium
- TestRailTest case and run management. webiosandroidapicommercial
- XrayTest management inside Jira. webiosandroidapicommercial
integration tests
- Charles ProxyHTTP proxy for inspecting and throttling traffic. iosandroidwebcommercial
- JestJavaScript test runner with mocking and snapshots. webapiopen-source
- JUnitTest framework for the JVM. apiandroidopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- pytestTest framework for Python with fixtures and plugins. apiopen-source
- VitestTest runner for Vite-based JavaScript and TypeScript projects. webapiopen-source
- XCTestApple's framework for unit and UI tests. iosopen-source
security scanning
- Burp SuiteProxy and scanner for web application security testing. webapifreemium
- GarakProbing LLM applications for prompt injection and leakage. apiopen-source
- mitmproxyInteractive HTTPS proxy. iosandroidwebapiopen-source
- MobSFStatic and dynamic analysis for mobile binaries. iosandroidopen-source
- OWASP ZAPDynamic application security scanner. webapiopen-source
- SchemathesisProperty-based testing from OpenAPI and GraphQL schemas. apiopen-source
- SemgrepStatic analysis with custom rules. apiwebiosandroidfreemium
- SnykDependency and code vulnerability scanning. apiwebiosandroidfreemium
- TrivyScanner for containers, dependencies and configuration. apiopen-source
device and browser compatibility
- AppiumCross-platform mobile UI automation over WebDriver. iosandroidopen-source
- ApplitoolsVisual comparison of screens across builds. webiosandroidcommercial
- BrowserStackCloud of real devices and browsers. webiosandroidcommercial
- Firebase Test LabRuns app tests on hosted devices. iosandroidfreemium
- LambdaTestCloud browser and device grid. webiosandroidcommercial
- PercyVisual review of UI changes. webcommercial
- PlaywrightBrowser automation for Chromium, Firefox and WebKit. webopen-source
- Sauce LabsCloud test infrastructure for web and mobile. webiosandroidcommercial
- SeleniumWebDriver-based browser automation. webopen-source
unit tests
- JestJavaScript test runner with mocking and snapshots. webapiopen-source
- JUnitTest framework for the JVM. apiandroidopen-source
- pytestTest framework for Python with fixtures and plugins. apiopen-source
- VitestTest runner for Vite-based JavaScript and TypeScript projects. webapiopen-source
- XCTestApple's framework for unit and UI tests. iosopen-source
contract tests
- PactConsumer-driven contract testing with a broker. apiopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- SchemathesisProperty-based testing from OpenAPI and GraphQL schemas. apiopen-source
regression suite
- CypressBrowser-based end-to-end testing with a visual runner. webfreemium
- DeepEvalUnit-style evaluation for LLM applications. apiopen-source
- PlaywrightBrowser automation for Chromium, Firefox and WebKit. webopen-source
- PromptfooEvaluation and regression tests for LLM prompts and outputs. apiwebopen-source
- TestRailTest case and run management. webiosandroidapicommercial
- XrayTest management inside Jira. webiosandroidapicommercial
resilience and failure testing
- Chaos MeshFault injection for Kubernetes. apiopen-source
- GremlinManaged fault injection. apicommercial
- ToxiproxyTCP proxy for simulating network failures. apiopen-source
performance and load testing
- ArtilleryLoad testing with YAML scenarios. apiwebfreemium
- GatlingLoad testing with Scala, Java or Kotlin scenarios. apiwebfreemium
- JMeterLoad testing tool from the Apache Foundation. apiwebopen-source
- k6Load testing scripted in JavaScript. apiwebopen-source
- LighthouseAudits for accessibility and performance in Chrome. webopen-source
- LocustLoad testing scripted in Python. apiwebopen-source
penetration testing
- Burp SuiteProxy and scanner for web application security testing. webapifreemium
offline and poor-network testing
- Charles ProxyHTTP proxy for inspecting and throttling traffic. iosandroidwebcommercial
- mitmproxyInteractive HTTPS proxy. iosandroidwebapiopen-source
- Network Link ConditionerApple tool for simulating poor network conditions. iosopen-source
- ToxiproxyTCP proxy for simulating network failures. apiopen-source
What to send to the person who will test
Send the checklist and the brief below. The printable checklist for SDK or client library has the same 9 items with checkboxes.
Download the checklistMarkdownCSVBrief (Markdown)
Acceptance criteria wording
A release of the SDK is accepted when it installs into sample host apps for every supported platform, builds from a clean cache and completes the quick-start call, and a network capture shows only the data types in its privacy manifest, with nothing sent before the consent call. The dependency report lists no known-malicious or high-severity package, and the sample host app keeps running when the SDK backend returns errors or malformed config.
Brief
# Test brief: SDK or client library Product type: SDK or client library. A library shipped to other developers' codebases. Risk map and evidence: https://howtotestaproduct.com/how-to-test/sdk/ Printable checklist: https://howtotestaproduct.com/checklists/sdk/ Run these 9 mandatory tests in this order. A test is done only when its done condition is true on the build you will release. 1. release smoke tests (risks: dependency-failure, store-rejection). Tool class: mobile automation. Usually run by: QA engineer. Done when: Each release candidate carries its privacy manifest and a signature that Xcode validates, installs into sample host apps for every supported platform, builds from a clean cache, launches and completes the quick-start call. 2. integration tests (risks: regulatory-penalty, data-breach). Tool class: proxy. Usually run by: developer. Done when: A proxy capture of a sample host app shows the SDK sends only the data types in its privacy manifest, and nothing before the host app's consent call. 3. compliance checks (risks: regulatory-penalty, store-rejection). Tool class: test management. Usually run by: product manager. Done when: The privacy manifest and the sample disclosure text list every data type the SDK collects and every recipient, signed off by legal, and the binary is signed. 4. security scanning (risks: data-breach). Tool class: dependency scanner. Usually run by: security specialist. Done when: The dependency report for the release lists no known-malicious or high-severity package, every transitive dependency is pinned in the lockfile, and maintainer changes are reviewed. 5. device and browser compatibility (risks: store-rejection, dependency-failure). Tool class: device cloud. Usually run by: QA engineer. Done when: The SDK builds and runs in sample host apps on the oldest and newest supported OS, language and package manager versions, and each iOS sample host app build passes App Store Connect validation with the SDK's privacy manifest. 6. unit tests (risks: dependency-failure). Tool class: unit framework. Usually run by: developer. Done when: In the test report, every server-sent config and response field in the fixed sample set passes with a null, wrong-type and missing value and no crash. 7. contract tests (risks: dependency-failure). Tool class: contract tool. Usually run by: developer. Done when: Before any server config change goes live, staging responses match the schema that every supported SDK version expects. 8. regression suite (risks: dependency-failure). Tool class: test management. Usually run by: QA engineer. Done when: The public API surface diff and the crash cases rerun on every release, and a removed or changed public method without a major version bump blocks the release. 9. resilience and failure testing (risks: dependency-failure). Tool class: chaos tool. Usually run by: developer. Done when: With the SDK backend returning errors, timeouts or malformed config, the sample host app keeps running and its own features work. Acceptance criteria wording: A release of the SDK is accepted when it installs into sample host apps for every supported platform, builds from a clean cache and completes the quick-start call, and a network capture shows only the data types in its privacy manifest, with nothing sent before the consent call. The dependency report lists no known-malicious or high-severity package, and the sample host app keeps running when the SDK backend returns errors or malformed config. Report back for each test: pass or fail against its done condition, the build or version tested, and the proof (screenshot, log line or report file). Licence: CC BY 4.0.
For founders: what to forward and what to ask back. For product managers: turning this page into tickets.
Questions about testing an SDK or client library
What testing does an SDK or client library need?
9 tests are mandatory, starting with a sample host app smoke run on every release candidate and a network capture of what the SDK sends. Each is tied to one of the 4 risks in the map: host app crashes after a server response change or an unpublished package, malicious code arriving through a transitive dependency, host apps rejected for a missing privacy manifest or signature, and sample privacy text that hides data recipients. Performance, penetration and offline tests are recommended, each with the event that makes it mandatory.
What goes on an API testing checklist for an SDK or client library?
Unit tests feed null, wrong-type and missing values for every server-sent config and response field. Contract tests check staging responses against the schema every supported SDK version expects before a server config change goes live. Resilience tests make the backend return errors, timeouts and malformed config while the sample host app keeps running.
Which tests in a vendor test plan for an SDK or client library are not needed?
Usability, accessibility and localization testing are not needed while the SDK draws no screens and shows no user-facing text, until it ships UI components or strings.
What is the minimum testing for an SDK or client library at MVP stage?
The minimum is the 9 mandatory tests once the SDK ships into other developers' builds, reads configuration from a server and has third-party dependencies. End-to-end runs of the public quick-start and exploratory sessions are optional at that stage.
When is test automation worth it for an SDK or client library?
Automation pays for checks that every release must pass before host apps pull it: the clean-cache build in sample host apps, the dependency report, the public API surface diff and the config parsing unit tests. Legal sign-off on the privacy manifest and sample disclosure text stays with people.
Evidence
ev-0297regulatorprimary
FTC Order Prohibits Data Broker X-Mode Social and Outlogic from Selling Sensitive Location Data, Federal Trade Commission
X-Mode/Outlogic sells and licenses precise location data that it collects from third-party apps that incorporate its software development kit (SDK) into their apps, from its own mobile apps, and by purchasing location data from other data brokers and aggregators.
Regulator order: the SDK maker gave host apps sample privacy disclosures that did not tell users how their location would be used. Supports regulatory-penalty and data-breach for mobile SDKs that collect location.
published 2024-01-09 · verified 2026-10-07
ev-0203post-mortemprimary
Details about the event-stream incident, npm
The injected code targets the Copay application.
Registry operator incident report: a new maintainer handed control of a widely installed package added a dependency carrying code aimed at one downstream build. Supports fraud-abuse for command-line tools whose install pulls third-party packages. The package was a library pulled into other projects' builds through their dependency tree, so it supports data-breach for an SDK whose transitive dependencies reach host app builds.
published undated · verified 2026-10-07
ev-0316regulatorprimary
Reported Supply Chain Compromise Affecting XZ Utils Data Compression Library, CVE-2024-3094, CISA
The malicious code may allow unauthorized access to affected systems.
Government alert: malicious code was embedded in released versions of a compression library, and users were told to downgrade. Supports data-breach for libraries distributed to other developers' systems.
published 2024-03-29 · verified 2026-10-07
ev-0204post-mortemprimary
Registry operator incident report: an author unpublishing a small package made installs and builds fail for every project that resolved it. Supports dependency-failure for command-line tools installed from a public registry. The unpublished package was a library that other projects resolved at install, so it supports dependency-failure for an SDK whose transitive dependencies fail host installs and builds.
published undated · verified 2026-10-07
ev-0299practitionerpractitioner
Suddenly FBSDKRestrictiveDataFilterManager.m - Line 71 started to crash, GitHub (facebook-ios-sdk issue tracker)
It suddenly started to crash from 14:44 PST 5/6/2020.
Named developer report: host apps crashed at launch inside the SDK after a server response changed, with no app release on either side. Supports dependency-failure for mobile SDKs that read remote configuration.
published 2020-05-06 · verified 2026-10-07
ev-0293store-guidelineprimary
Third-party SDK requirements, Apple
As a reminder, when you use a third-party SDK with your app, you are responsible for all the code the SDK includes in your app, and need to be aware of its data collection and use practices.
Store guideline: host apps submitted to App Store Connect must include the privacy manifest for each listed SDK, and Xcode validates SDK signatures. Supports store-rejection for mobile SDKs directly, not by transfer.
published undated · verified 2026-10-07
Disclaimer
This page lists tests and risks for a product category, not for your product. Risks are attached only with the evidence shown in the Evidence section. Tools are listed alphabetically and are not recommendations. Nothing here is legal, regulatory or security advice; where a regulator or a store sets a requirement, their page is linked and takes precedence.