Chain of Custody Verification

The problem

A deletion request crosses at least six handoffs: interface, API, backend, external party, stored state, and the interface again. Feature-level tests check each layer against its own specification, and take the system's own status as the oracle for whether the work was done.

When a failure happens between layers rather than inside one, every layer passes its own test. Every test is green. The user sees "Removed" and nothing was removed. The defect is invisible precisely because the signal that would report it is the signal that is wrong.

This class of defect matters more in privacy and security software than almost anywhere else, because the user cannot check the claim themselves. A person who is told their personal data was deleted from a broker has no way to confirm it, and adjusts their behaviour as though the protection were real.

The chain

CoCV follows one piece of user data across every handoff it makes, from the user's action to the user's confirmation, and compares adjacent layers against each other and against downstream reality. Two rules do most of the work.

First, a layer's status is never accepted as evidence about another layer's behaviour. A collector reads the state its own layer actually holds, and never a status that some other layer derived. Second, the interface is checked last and distrusted first: what the user is shown is the claim under test, not the standard against which the rest is measured.

The chain itself is declared rather than coded: which layers exist, which handoffs connect them, which verification points apply to each handoff, and which fields must survive a handoff unchanged.

Verification points

Ten checks are applied across adjacent layers. Not every point applies to every handoff; the chain definition declares which do.

When a check fails, the reconciler localizes the first broken link rather than reporting a diffuse failure. The output names the handoff, the verification point and the divergence.

Applying CoCV to your own product

1. Pick one protective claim your product makes to the user. Not a feature: a claim that something happened.
2. Draw the chain. List every layer the user's data crosses between the user's action and the user's confirmation, including external parties.
3. Write the chain definition. Declare which verification points apply to each handoff and which fields must survive each handoff unchanged.
4. Write a collector per layer. A collector reads that layer's real state for one operation and returns an observation. It must not read a status another layer derived.
5. Reconcile and read the trace. For the first claim, do this by hand before automating anything.
6. Only then wire it into continuous integration and, if useful, attach a classifier.

Where AI fits

The reconciler decides whether and where the chain broke. That decision is deterministic and involves no model. A classifier then explains the divergence: likely root cause, user impact, what to check first.

A rule-based classifier is always available and makes no external calls. A language-model classifier is optional and accepts any caller-supplied completion function, so no provider is hard-coded and no credentials are read by the library. The sequence is deliberate: the architecture determines what is verified, and a model is applied to the result, never asked to decide what to check.

Demonstration and evidence

The reference implementation includes a synthetic six-layer removal service with seven injectable faults, each reproducing a silent protection failure pattern observed in real systems: an empty external response mapped to success, an address trimmed by the API layer, one broker failing while the aggregate status reads complete, a request queued and recorded as sent, a payload logged in cleartext, a record reappearing after deletion, and telemetry emitted from the same status field the interface reads.

For every one of these faults, the conventional feature-level check passes and CoCV fails and localizes the break. That pairing is the point of the demonstration: it is a reproducible statement of what the method catches that existing practice does not.

Scope and limits

CoCV applies where a product makes a protective claim the user cannot independently verify, and where fulfilling that claim crosses system or organisational boundaries. Data removal, secure erasure, credential protection and consent propagation are the clearest cases.

It is not a replacement for functional or security testing, and it does not find defects contained entirely within one layer. It costs more than feature-level testing, because it requires a collector per layer and access to real state. That cost is justified for claims where a silent failure leaves the user worse off than no feature at all, and is hard to justify for anything else.

Relation to published standards

CoCV verifies attributes that international standards already define as required, using a technique those standards do not supply. ISO/IEC 25010 names functional correctness, integrity and accountability among the required quality characteristics of a software product. ISO/IEC 25012 names accuracy, consistency and traceability among the required quality characteristics of data. ISO/IEC/IEEE 29119 defines the processes and documentation by which software is tested, and catalogues the techniques available for designing tests.

None of the three supplies a technique for verifying that a protective operation on personal data completed end to end across independent system layers, which is the condition on which those characteristics depend in this category of software. CoCV conforms to the process and documentation model of ISO/IEC/IEEE 29119, verifies characteristics named in ISO/IEC 25010 and ISO/IEC 25012, and supplies the technique those standards assume but do not define. The full mapping, verification point by verification point, is published with the reference implementation.

Status and independence

Version 0.1.1, public release. Complete and tested, and open for review and adoption.

This methodology and its implementation were developed independently by the author, using no code, data or internal materials of any employer. The synthetic service in the reference implementation is invented for demonstration and does not model any specific product.

Citation and licensing

The methodology text is published under CC BY 4.0. The reference implementation is released under the MIT licence. Both permit commercial use, adaptation and redistribution with attribution.