Green Tests, Broken Crate

Four times in four days, the thing we tested was not the thing we shipped

Audience: Rust library maintainers, and anyone who publishes to crates.io Version: 1.0 | September 2026

The pattern

In the first week of September 2026 we took four of our Rust crates through a release review. Each one looked fine from where its author was standing. Each one had a real defect that the author's own checks could not see.

The four bugs were unrelated, but they had one cause: the check ran on a different thing from the one a user would get. Here they are, and the rule we took from each.

1. cargo test passed. cargo build failed.

Our privacy crate (now pqc-privacy) had a green test suite, and a plain cargo build of the library failed.

The crate listed rand under [dev-dependencies] with its default features on. Cargo unifies features across the dependency graph, so the test build got a rand with thread_rng enabled, and code that quietly relied on it compiled. The library build, which is what cargo add gives a user, has no dev-dependencies. It never got that feature and did not compile.

cargo test and cargo build resolve different dependency graphs. Green tests are not evidence that the library compiles.

Rule: run cargo build next to cargo test, always.

2. The crate built everywhere. You couldn't depend on it.

pqc-kem's CI was thorough: fresh clone, minimum Rust version, every feature combination, and the packaged artifact. It still missed that the crate could not be used as a no_std dependency and had no working wasm32 build, which are both things the README promised.

Both failures only exist in a consumer's build graph: a lang-item collision, and two major versions of getrandom pulled in side by side. Building the crate on its own, in any configuration, cannot show them.

Rule: test a capability the way a user exercises it. pqc-kem's CI now builds a separate little crate that depends on pqc-kem as an ordinary no_std library for wasm32. That job is the regression test for this bug.

3. The README described a component we never built.

Our gossip crate, witan, was described as a WebAssembly Component Model component, with a WIT interface to match. What the build produced was a WASI Preview 1 core module with a hand-rolled pointer-and-length ABI. The WIT file was accurate and entirely disconnected from the build.

A native build or dry-run can't catch that, because the component code sits behind a target-specific cfg that never compiles on the host.

Rule: build the target you claim. witan 0.2 builds a real wasm32-wasip2 component, and CI builds it.

4. It linked on Windows. It could not be published.

Our NATS transport crate is a cdylib whose exports come from a WIT interface. The export symbol names contain : and #. The Windows linker accepts them; GNU ld and lld reject them in a version script.

cargo publish verifies a package by building it for the host. On Linux that build failed, so the crate could not be published from Linux at all, however the tests ran on the author's machine.

Rule: build the packaged artifact, not the working tree. cargo package unpacks the .crate and builds it the way crates.io will. It also catches a path dependency that has no version, which was a second bug in the same crate.

The quiet version of the same bug

Three weeks later we found a subtler variant in our own test suite. Several test files were gated with a file-level #![cfg(feature = "kat")]. Run without the feature, each one compiles to an empty test binary that reports "0 passed" as a success. A default cargo test printed five of them. Nothing failed; the tests just weren't there.

The fix is required-features on each [[test]] target in Cargo.toml. Cargo then skips those targets without the feature, and asking for one explicitly without it is an error rather than a silent pass.

The checklist we use now

1. cargo build next to cargo test, every time.
2. Build every target the README claims (wasm32-unknown-unknown, wasm32-wasip2, no_std), not just the host.
3. A separate consumer crate in CI that depends on the library the way a user would.
4. cargo package and test the unpacked .crate, not the working tree.
5. required-features for feature-gated tests, never a file-level cfg.
6. Lockfiles checked with --locked in CI, so drift fails loudly.

None of these are exotic. We needed four bugs in four days to start doing all six.

This is the release bar we are bringing every 0x307 crate up to before it ships. The crates, their tiers and their audit status are at 0x307.com/crates.