SKILL·B10512

property-based-testing

trailofbits
Updated 14 days ago
35 views
7,515
642
7,515
View on GitHub
Metaaitestingapidesigndata

About

This skill writes, reviews, and debugs property-based tests for frameworks like Hypothesis and fast-check, generating broad input domains instead of single examples. It's ideal for testing serialization, data structures, validators, and smart contract invariants, or for analyzing shrunken counterexamples. It is not suited for binary fuzzing, mutation testing, or UI tests.

Quick Install

Claude Code

Recommended
Primary
npx skills add trailofbits/skills -a claude-code
Plugin CommandAlternative
/plugin add https://github.com/trailofbits/skills
Git CloneAlternative
git clone https://github.com/trailofbits/skills.git ~/.claude/skills/property-based-testing

Copy and paste this command in Claude Code to install this skill

Documentation

Property-Based Testing

An example test asserts one point. A property asserts a rule over the whole input domain and lets the generator hunt for the counterexample. That trade is worth making when the code has an algebraic shape — an inverse, an invariant, an oracle — and not otherwise. Code with no such shape gets example tests; saying so is a valid outcome.

Check first whether the shape is missing or merely buried. A calculation wrapped in I/O, a string built by concatenation, an in-place mutation — each has a property and no seam to assert it through. See references/refactoring.md before concluding there is nothing to assert.

Property catalog

PropertyFormulaWhere it applies
Roundtripdecode(encode(x)) == xSerialization, conversion pairs
Inversef(g(x)) == xencrypt/decrypt, compress/decompress
Oraclenew(x) == reference(x)Optimization, refactoring, reimplementation
Idempotencef(f(x)) == f(x)Normalization, formatting, sorting
InvariantHolds before and afterAny transformation, contract state
Easy to verifyis_sorted(sort(x))Complex algorithms with cheap checkers
Commutativityf(a, b) == f(b, a)Binary and set operations
Associativityf(f(a,b), c) == f(a, f(b,c))Combining operations
Identityf(x, e) == xOperations with a neutral element

Strength ordering, weakest to strongest: no crash → type preservation → invariant → idempotence → roundtrip / oracle.

Assert the strongest property the code supports. "No crash" alone rarely justifies the dependency — if that is all you can find, either a small rearrangement exposes something stronger, or the honest report is that this code is a poor PBT candidate. Rule out the first before settling for the second.

The two ways a property test asserts nothing

  • Tautology. assert add(a, b) == a + b restates the implementation; no bug they share can fail it. Pick a property that constrains the function without recomputing it. Note the exception: f(x) == f(x) is a genuine determinism property when f is not obviously pure — serializers over dicts or sets, hashing, anything reading the clock.
  • Vacuity. assume() that filters out nearly every input passes without exercising anything, and self-contradictory assume() passes having run zero cases. Push constraints into the strategy so the generator produces valid inputs directly.

Where to look next

Load the one that matches the task in front of you:

TaskFile
Writing new tests, designing strategiesreferences/generating.md
The code has no property to assert yetreferences/refactoring.md
Reviewing existing property testsreferences/reviewing.md
A property test just failedreferences/interpreting-failures.md
Library choice, Echidna and Medusareferences/libraries.md

Introducing PBT to a project that lacks it

If the project already uses a PBT library, just write the tests in it. If it does not, adding one is a dependency decision that belongs to the user — offer it once with the specific property you would write, and take the answer either way.

GitHub Repository

trailofbits/skills
Path: plugins/property-based-testing/skills/property-based-testing
0
agent-skills
FAQ

Frequently asked questions

What is the property-based-testing skill?

property-based-testing is a Claude Skill by trailofbits. Skills package instructions and resources that Claude loads on demand, so Claude can perform property-based-testing-related tasks without extra prompting.

How do I install property-based-testing?

Use the install commands on this page: add property-based-testing to Claude Code as a plugin, or clone its repository into your skills directory, then restart Claude so it picks up the skill.

What category does property-based-testing belong to?

property-based-testing is in the Meta category, tagged ai, testing, api, design, and data.

Is property-based-testing free to use?

Yes. property-based-testing is listed on AIMCP and free to install.

Related Skills

content-collections
Meta

This skill provides a production-tested setup for Content Collections, a TypeScript-first tool that transforms Markdown/MDX files into type-safe data collections with Zod validation. Use it when building blogs, documentation sites, or content-heavy Vite + React applications to ensure type safety and automatic content validation. It covers everything from Vite plugin configuration and MDX compilation to deployment optimization and schema validation.

View skill
polymarket
Meta

This skill enables developers to build applications with the Polymarket prediction markets platform, including API integration for trading and market data. It also provides real-time data streaming via WebSocket to monitor live trades and market activity. Use it for implementing trading strategies or creating tools that process live market updates.

View skill
creating-opencode-plugins
Meta

This skill helps developers create OpenCode plugins that hook into 25+ event types like commands, files, and LSP operations. It provides the plugin structure, event API specifications, and implementation patterns for JavaScript/TypeScript modules. Use it when you need to intercept, monitor, or extend the OpenCode AI assistant's lifecycle with custom event-driven logic.

View skill
sglang
Meta

SGLang is a high-performance LLM serving framework that specializes in fast, structured generation for JSON, regex, and agentic workflows using its RadixAttention prefix caching. It delivers significantly faster inference, especially for tasks with repeated prefixes, making it ideal for complex, structured outputs and multi-turn conversations. Choose SGLang over alternatives like vLLM when you need constrained decoding or are building applications with extensive prefix sharing.

View skill