After an innovation experiment: when should we scale, adapt or stop?

An innovation experiment ends with encouraging results: an indicator improves, participants respond positively, and the prototype becomes usable. The decision meeting may reveal a more complicated picture. Some users benefited less, delivery required exceptional support, or the cost of expansion remains unclear. The organization needs a decision that reflects what the evidence actually supports and what still needs to be learned.

This article addresses the stage after an experiment. It proposes a decision card for innovation teams and business leaders: expand within a defined scope, adapt and retest, or stop while preserving learning. The card should be tailored to the experiment’s size and risks; it is not an official standard or a set of universal thresholds.

Return to the question the experiment was designed to answer

Open the meeting with the hypothesis and decision criteria recorded before testing, then place the results beside them. Was the aim to test understanding, reduce completion time, or establish whether delivery was feasible? An experiment may answer only one question. The UK government’s Test, Learn, Adapt guidance emphasizes specifying the intended outcome in advance when designing evaluation. [1]

Preserve the version tested, delivery period, participant groups, sample size and data source. Document exclusions and changes during delivery. If a definition or measurement method changed, explain its effect on comparability. Keep the original threshold visible: a new observation may be useful without retrospectively redefining success for the completed experiment.

Read user and operational results together. Fewer enquiries could mean clearer instructions or reduced access to help. Examine complaints, incomplete cases and staff workload. Distinguish observed results, possible explanations and unanswered questions so the decision owner can see what the team knows and what it assumes.

Assess the evidence and its limits in a different setting

Nesta’s Standards of Evidence distinguish observing positive change from showing that an intervention caused it, and from demonstrating repeatable impact. [2] A before-and-after comparison alone therefore does not establish causation: demand, case characteristics or staff experience may have changed over the same period.

Ask an evaluation specialist to determine suitable comparisons and data requirements for the question and risks. A small experiment may reveal a usability barrier without establishing an overall impact. Where evidence is inconclusive, record that explicitly and identify the gap. An extension needs a clear question that additional evidence could answer.

Examine the conditions behind the result. Did the idea’s owner supervise every case? Were participants intensively trained? Was extra support available? Record which conditions would remain during expansion. Review relevant user groups too: a favorable average may conceal difficulties, while small numbers cannot establish reliable differences between groups.

RMG S14 Internal 01 EN V01

Read experiment results alongside their original criteria and evidence limitations before choosing the next step.

Choose a next step that matches what has been learned

The UK Test and Learn guidance connects expansion decisions to understanding impact and the conditions in which an intervention works. Findings may support local adaptation or reconsideration. [3] The following table proposes a practical application, helping a meeting produce a defined action rather than a general commitment to continue development.

Decision When appropriate What to authorize
Bounded expansion Evidence supports the proposed step, its conditions are met, and no critical barrier remains. A next user group or location, specified resources, monitoring and a review date.
Adapt and retest Potential value remains, with an addressable barrier or a specific evidence gap. A defined change, test question, time and cost limits, and new decision criteria.
Stop Evidence weakens the core hypothesis, harm is unacceptable, or no viable path remains. An orderly close, action on consequences, and a record of learning and reasons.

For expansion, separate the unusually supported pilot cost from recurring delivery costs: staff time, training, support and maintenance. For adaptation, identify the assumption being changed and how to test the change. Stopping can preserve useful knowledge for another experiment; previous expenditure alone is insufficient reason to commit more resources.

A high overall score cannot offset a critical barrier. If harm triggers an organizational stopping rule, act through the designated owner without waiting for the final meeting. A temporary suspension for investigation or correction differs from closing the experiment permanently. Record the distinction and any conditions for resuming.

Use a decision card that can be reviewed later

Copy and complete this card before authorizing the next step. Record disagreements and the evidence that could resolve them rather than hiding them within a general approval.

Card field Complete in the experiment record
Experiment and hypothesis ID and version: ___ | Hypothesis tested: ___
Criterion and result Original decision criterion: ___ | Actual result: ___
Evidence and limitations Evidence link: ___ | Sample, setting and unresolved questions: ___
Risks and costs Remaining barrier or risk: ___ | Next-step cost and resources: ___
Decision and scope Decision and rationale: ___ | Permitted scope: ___ | Reversal condition: ___
Ownership and follow-up Decision owner: ___ | Action and responsible person: ___ | Review date: ___

Separate responsibilities: the experiment owner explains delivery, a data specialist checks interpretation, the service owner assesses practical application, and the authorized decision maker approves the next step under organizational governance. Small teams may combine roles while keeping responsibilities explicit. The card should point to evidence without reproducing every laboratory activity.

Hypothetical example: an improvement that does not justify broad rollout

Suppose an organization tests a checklist for checking application completeness before referral. All figures are fictional, not client results or recommended thresholds. The team predefines a reduction in returns for missing information from 20% to 12% or less, no more than one additional minute of average review time, and no acceptance of an application missing a mandatory requirement.

Returns reach 10%, review time increases by half a minute, and no noncompliant acceptance is observed in the sample. Yet all cases were simple and reviewed by experienced staff. There was also no comparison capable of separating the checklist’s effect from changes in case mix. The proposed decision is another bounded test across more varied cases, with an appropriate comparison designed by a specialist.

The card records encouraging observations and insufficient evidence for broad expansion, together with a resource limit and another decision date. If review time exceeded its limit, a specific modification would need testing. If an incomplete application were accepted, the agreed suspension and corrective process would apply. Each result leads to a defined action.

RMG S14 Internal 02 EN V01

Each path ends with an action, an owner and a review point, while preserving evidence and learning.

Each path ends with an action, an owner and a review point, while preserving evidence and learning.

Make experiment decisions part of laboratory operations

Schedule the results review when launching the experiment and retain the card with its evidence and tested version. Follow up the agreed action and record how learning changed the work plan. Nesta emphasizes clear evaluation questions and limitations when scaling innovations, alongside attention to unintended harm. [4]

RMG’s innovation lab design, build and operation service includes needs assessment, laboratory management, training, support and innovation programs. [5] Start a practical discussion with RMG using an existing experiment’s results, its original criteria and the decision your organization needs to resolve, then define a scope that connects experimentation with business needs.