Document Type: Framework
Status: Active
Version: v1.0
Authority: MWMS HeadOffice
Applies To: Experimentation Brain, Research Brain, Affiliate Brain, Finance Brain
Parent: Experimentation Brain
Last Reviewed: 2026-04-03
Purpose
This page defines the criteria used to determine whether experimental evidence is considered valid for decision-making.
Evidence validation ensures:
- decisions are based on reliable signals
- weak or noisy data is not over-interpreted
- scaling decisions are supported by credible learning
- signal interpretation remains consistent across experiments
- testing discipline remains structured
Core Principle
Evidence must be interpretable before it can be actionable.
Unvalidated evidence increases decision risk.
Validation Criteria Dimensions
Criterion 1 — Data Integrity
Evidence must be based on reliable data collection.
Examples:
tracking functioning correctly
conversion signals correctly recorded
click data correctly captured
no major tracking interruptions
Poor data integrity reduces confidence in interpretation.
Criterion 2 — Variable Isolation
Experiment design must isolate meaningful variables.
Examples:
only one primary variable changed at a time
test structure clearly defined
variables clearly understood
Multiple simultaneous changes reduce interpretability.
Criterion 3 — Sample Adequacy
Evidence must be based on sufficient observation volume to reduce randomness effects.
Small sample sizes increase noise risk.
Adequate sample volume improves reliability of interpretation.
Criterion 4 — Behaviour Consistency
Observed behaviour should demonstrate repeatability.
Examples:
similar audience behaviour patterns
consistent response to similar creative structures
stable performance indicators across comparable tests
Consistency increases confidence strength.
Criterion 5 — Hypothesis Alignment
Observed outcomes should logically relate to the defined hypothesis.
Hypothesis mismatch reduces learning clarity.
Evidence must inform decision logic.
Criterion 6 — Signal Stability
Observed signals should remain reasonably stable across time periods.
High volatility reduces interpretability.
Stable signals support confidence progression.
Criterion 7 — Noise Awareness
Observed variation should be explainable or bounded.
Unexplainable volatility reduces reliability of conclusions.
Noise awareness prevents premature interpretation.
Criterion 8 — Interpretation Clarity
Evidence should allow a clear conclusion about:
what changed
why behaviour changed
what learning occurred
Unclear interpretation reduces learning value.
Evidence Validation Outcomes
Validated Evidence
Signals considered reliable for decision progression.
Partially Validated Evidence
Signals provide directional insight but require further testing.
Unvalidated Evidence
Signals unreliable or too ambiguous for decision confidence.
Evidence Discipline Principle
Evidence validation protects both:
learning quality
capital stability
Relationship to Other Pages
Experimentation Signal Strength Classification
Experimentation Confidence Progression Model
Finance Brain Capital Confidence Thresholds
Research Brain Evidence Tier Classification
Experimentation Brain Canon
Architectural Role
This page defines the minimum quality standard for experimental learning signals used by MWMS.
Future Expansion
Future versions may include:
evidence scoring systems
signal reliability models
automated validation indicators
evidence dashboards
Change Log
Version: v1.0
Date: 2026-04-03
Author: MWMS HeadOffice
Change: Initial creation of Experimentation Evidence Validation Criteria defining reliability standards for experiment evidence.