# Property-Based Testing

> Property-based testing generates many inputs and checks the rules that should hold across them.

- HTML version: https://robbiepalmer.me/ideas/property-based-testing
- Source: https://fast-check.dev/docs/introduction/getting-started/

Example-based tests choose an input and state its expected output. Property-based
tests describe a wider rule, generate many inputs within a defined space, and
look for an input that breaks the rule. Both forms matter. Examples document
known cases well, while properties search combinations that a person did not
think to write down.

A property is a claim about behaviour rather than one expected value. Parsing
arbitrary text should not crash. Serializing and then parsing a value should
preserve its meaning. Normalizing an already normalized value should make no
further change. Converting between compatible units and back should recover the
original quantity within the stated precision.

The input generator is part of the test. It defines which values can appear,
how large they may become, and how often important shapes occur. A generator
that produces only easy values can make a strong-looking property almost
empty. A generator that recreates the production implementation can repeat the
same mistake on both sides of the assertion.

## Shrinking makes failure useful

When a generated input fails, a property-testing library searches for a smaller
input that still fails. A parser failure found in a large document may shrink to
one malformed token. A conversion failure may shrink to the smallest quantity
or unit pair that crosses a boundary.

The smallest counterexample is easier to understand and preserve as a
focused regression test. The property should remain too. The example protects
the known defect, while the property continues searching nearby and unrelated
inputs.

Shrinking does not prove that the final example is the only cause. Its result
depends on the generator, the shrink order, and any values discarded by
preconditions. Treat the counterexample as evidence to investigate rather than
an explanation generated by the tool.

## Reproducibility needs an explicit boundary

Generated tests still need repeatable failures. A run should report the random
seed and the path through the shrink process. Replaying both values should
produce the same counterexample against the same code and dependency versions.

Routine suites also need bounds. Limit collection sizes, string lengths,
recursion depth, case counts, and total runtime. A larger exploratory run can
exercise more combinations outside the pull-request path. A fixed routine
budget keeps the test useful in ordinary development instead of turning every
change into a fuzzing campaign.

## Properties are executable laws

Round trips, idempotence, equivalence, ordering, conservation, and domain
invariants make good properties because they state what must remain true across
many examples. Safety properties such as "this parser does not throw" are also
valuable, but they say less about whether a successful result is correct.

[Falsifiability](/ideas/falsifiability) gives the basic shape. A useful property
rules out observable counterexamples. [Critical rationalism](/ideas/critical-rationalism)
adds the caution that surviving generated cases corroborates the rule without
proving it for every possible input.

[Mutation testing](/ideas/mutation-testing) challenges the program by changing
its implementation. Property-based testing keeps the implementation fixed and
varies its inputs. Used together, generated inputs can expose a mutant that a
small example suite misses, while surviving mutants can show that a property is
too weak or its generator never reaches the relevant case.

Property-based testing pays for itself when the domain has clear laws and a
large input space. The method fits poorly when correctness depends mainly on
visual judgement, external systems, or examples that need a copy of the
implementation to calculate their expected result.

## Related ideas

- [Critical rationalism](https://robbiepalmer.me/ideas/critical-rationalism.md): Knowledge grows through conjecture and criticism, with claims kept open to refutation rather than justified as certain.
- [Falsifiability](https://robbiepalmer.me/ideas/falsifiability.md): A claim is falsifiable when some possible observation could count against it, though real tests examine a bundle of assumptions.
- [Mutation Testing](https://robbiepalmer.me/ideas/mutation-testing.md): Mutation testing changes program behaviour on purpose and checks whether the test suite notices.

## Where it appears

- Technology: [fast-check](https://robbiepalmer.me/technologies/fast-check.md)
- ADR: [ADR 046: Property-based testing for TypeScript domain packages](https://robbiepalmer.me/projects/personal-engineering-platform/adrs/046-property-based-testing-for-typescript-domain-packages.md)

---

Markdown index of this site: https://robbiepalmer.me/llms.txt