Measurement
Part of Lead network measurement that preserves the state ledger
Eight fixtures to test lead network attribution methods before reporting
Eight authorised fixtures that test a lead network attribution rule before any report, with the expected result recorded first and a pause rule.
A lead network attribution rule is trustworthy only once it has produced a known answer on cases built to break it. Eight authorised fixtures do that, each forcing one decision the rule must make. Record the expected assignment first, then run the report and compare the two.
What to take away
- Test attribution methods on authorised fixtures, not on live records.
- Record the expected assignment before the run, then compare.
- A mismatch pauses the report until identity, window and deduplication logic are corrected.
- A passing test shows the rule runs as written, not that the network caused the outcome.
What a fixture test shows, and what it does not
A fixture test checks that a rule behaves as written. It cannot show that the network caused an outcome. An attribution total can reconcile to recorded outcomes and still overstate causal contribution, because other media, seasonality, price or customer intent may have produced the result.
Keep the two layers apart in the report. Attribution accounts for recorded touchpoints under a named rule. Causal evaluation asks what would have happened without the network, and it needs its own design. Choose rule-based attribution for accounting, and causal evaluation only when an incremental decision justifies it.
Attribution or Incrementality?
Does the decision concern accounting for recorded touchpoints?
use rule-based attribution
does an incremental decision justify causal design?
The HM Treasury Magenta Book is central government guidance on evaluation, and its Annex A gives detailed guidance on the range of evaluation methods. A private buyer should choose a proportionate design with a qualified evaluator rather than copy a method label. That is separate work from fixture testing.
Write the rule before the fixtures run
Name the eligible source event, outcome event, identity key, attribution window and time zone before any fixture runs. A fixture can only fail against a rule that was written down first. Say whether a credit, cancellation or refund revises an earlier assignment.
Also write how retries, duplicate enquiries, cross-device activity, offline contact and multiple recipients are treated. Freeze the query and rule version for the reporting period, so a later result can be traced to the rule that produced it.
Specify the Assignment Rule
- Name eligible source event and outcome event
- Name identity key, window and time zone
- Explain retries, duplicates, cross-device, offline contact
- Preserve all nine record states in reporting
- Run data-loss checks before publishing
- Freeze query and rule version per period
The eight fixtures
Each fixture isolates one decision the rule must make. Build them from authorised test data, and write the expected assignment next to each before the run.
| Fixture | What it contains | Decision it forces |
|---|---|---|
| 1. One source | A single source event and one outcome | The baseline: every method should agree |
| 2. Repeated source events | The same source arriving several times | Whether a retry or duplicate enquiry earns a second credit |
| 3. Two channels | Two sources before one outcome | First-source, last-source or fractional split |
| 4. Two devices | One person on two devices | Whether the identity key joins them |
| 5. Late offline outcome | An outcome recorded after the window closes | Window, time zone and late arrivals |
| 6. Rejection | A record rejected after assignment | Whether it stays visible in operational reporting |
| 7. Credit | A credit issued after the outcome | Whether earlier assignment is revised |
| 8. Cancellation | A cancelled outcome | Whether earlier assignment is revised |
Rejected and quarantined fixtures must still show in operational reporting. That is what the state ledger is for: assigning a fulfilled service to a source does not let the others vanish.
Run each fixture through each method
Run the same eight fixtures through first-source, last-source and fractional multi-touch assignment, then compare every result with the expected one. The methods answer different questions and fail in different ways, so the rule must be named before any share is compared.
Attribution Approaches Compared
first eligible source
- Question answered
- which recorded source began journey
- Principal limitation
- later influences ignored
last eligible source
- Question answered
- which qualifying source preceded outcome
- Principal limitation
- earlier activity disappears
fractional assignment
- Question answered
- how credit divided by rule
- Principal limitation
- weights are assumptions
causal evaluation
- Question answered
- what changed vs counterfactual
- Principal limitation
- design assumptions may fail
First-source and last-source reports usually count assigned outcomes, so the denominator must state which outcomes were eligible for assignment. A fractional report needs weights that sum under a reproducible rule. Never compare shares if one method excludes unmatched outcomes and another retains them.
Run the fixtures as a path audit before choosing a rule, not after the first report has gone out.
When a fixture fails
A mismatch pauses the report until identity, window and deduplication logic are corrected. Do not publish a figure from a rule that failed a fixture, and do not edit the expected result to match the output.
Run data-loss checks before publishing. Report missing source events, unmatched outcomes, late arrivals and cross-channel exposure that the system cannot see. Preserve the submitted, quarantined, validated, matched, disclosed, accepted, rejected, credited, contacted and fulfilled states throughout.
Keep fixtures inside the data boundary
Fixtures should carry the minimum personal data the test needs. Under the UK GDPR, personal data must be adequate, relevant and limited to what is necessary for the purposes for which it is processed.
Identity stitching and longer windows can increase apparent coverage while increasing privacy and security risk. Purpose, minimisation, UK GDPR role and PECR channel decisions remain separate from analytical convenience. Use aggregate outputs where possible and restrict case-level joins to authorised users.
Protect the Data Boundary
- Keep purpose and minimisation separate from convenience
- Confirm UK GDPR role and PECR channel decisions
- Use aggregate outputs where possible
- Restrict case-level joins to authorised users
- Report both layers under their real names
Before you act
- Write the rule elements down before any fixture runs.
- Record the expected assignment for all eight fixtures first.
- Run every fixture through each method under comparison.
- Pause the report on any mismatch until the logic is corrected.
- Keep case-level joins to authorised users.
- Do not relabel attribution as incrementality.
Common questions
Why record the expected assignment first?
Otherwise the output can be argued into the expected result. Written first, a mismatch is a finding, and it pauses the report until identity, window and deduplication logic are corrected.
Does a passing fixture test prove the network works?
No. It shows the rule runs as written. An attribution total can reconcile to recorded outcomes and still overstate causal contribution, so a causal claim needs a separate design.
What data should the fixtures contain?
Authorised test records holding the minimum personal data the test needs, with aggregate outputs where possible. Case-level joins stay restricted to authorised users.
