Core Concepts

Data hierarchy

Domain (your site)
  └── Test Run (a URL being tested)
       └── Test (one element being optimized)
            └── Variant (a copy alternative)

• Domain: Your website (e.g., mysite.com). You authenticate with a domain-level API key.
• Test Run: A specific page URL. Each run can have up to 5 concurrent tests.
• Test: Targets a single element on the page (headline, CTA, description). Has a control and multiple variants.
• Variant: A specific copy alternative. The SDK applies it to the page via DOM manipulation.

Thompson Sampling

Endless Testing uses Bayesian bandits (Thompson sampling) instead of fixed traffic splits:

• Each variant has a Beta distribution based on its conversion history
• On each page load, the SDK samples from each variant’s distribution
• The variant with the highest sample “wins” and gets shown
• This naturally allocates more traffic to better-performing variants
• No waiting for a fixed sample size — the system adapts continuously

Auto-evaluation

Tests are evaluated automatically when:

• Enough impressions and conversions have accumulated
• Statistical confidence thresholds are met
• A winner is clear

When a winner is detected:

1. The winning variant’s copy becomes the new control
2. AI generates new challenger variants
3. The cycle repeats

Kill Switch

Emergency stop that instantly pauses all tests on a domain. Available via:

• CLI: endlesstesting stop
• Dashboard: big red button
• MCP: stop_test tool

No variant changes are served while the kill switch is active.

Brand Context

AI-generated variants respect your brand guidelines:

• Brand name and description
• Tone of voice
• Allowed/forbidden claims
• Claim safety rules (flag exaggerated or unverifiable claims)

Configure via endlesstesting init or update later through the API.