THE SEARCH LAB / FIELD NOTES

The Experiment Archive

What I tested. What I found. What I learned.

5 experiments

01Content optimisation / Answer Engine Optimisation

Can clearer question-led content improve how a service page answers user queries?

Planned · Proposal
Published status: PlannedProposed experiment · not yet tested

The Question

Can clearer question-led content improve how a service page answers user queries?

The Hypothesis

I expect that directly answering the questions buyers ask could make the page easier to understand. That would not, by itself, establish a ranking or citation benefit.

The Method

  1. I would list the customer questions the existing page needs to answer and record a baseline version.
  2. I would revise the structure and answers while keeping the offer consistent, then ask comparable readers to complete the same understanding tasks.
  3. I would compare answer accuracy and task completion, and document other changes or differences between the reader groups. Search observations would be a separate, exploratory measure.

The Evidence

I plan to retain the before/after copy, the task prompts and anonymised assessment notes. No reader sessions or search observations have been collected.

The Outcome

Not tested. There is no observed outcome to report.

The Learning

Next investigation: Before testing, I would define what counts as a clear, accurate answer and decide how many reader sessions are feasible.

02Technical SEO / Content optimisation

When should overlapping landing pages be consolidated?

Planned · Proposal
Published status: PlannedProposed experiment · not yet tested

The Question

When should overlapping landing pages be consolidated?

The Hypothesis

I expect consolidation may help where two pages serve the same need, but keeping pages separate may be more useful when their intent or audience differs.

The Method

  1. I would map each candidate page to its queries, audience, purpose and existing useful content.
  2. I would review available links, conversions and historical visibility before choosing a merge, differentiation or no-change approach.
  3. For any merge, I would document content retained, redirects and internal-link updates, then compare a defined follow-up period with the baseline and relevant unchanged pages.

The Evidence

I plan to retain a query-to-page map, the decision rationale, implementation checks and dated reports where available. No consolidation or measurement has been performed.

The Outcome

Not tested. There is no observed outcome to report.

The Learning

Next investigation: I would investigate whether any lost query coverage reflects a distinct need that should have been preserved. Seasonality and wider site changes would limit causal conclusions.

03AI Search / GEO / Answer Engine Optimisation

How consistent are brand mentions across repeated AI search observations?

Planned · Proposal
Published status: PlannedProposed experiment · not yet tested

The Question

How consistent are brand mentions across repeated AI search observations?

The Hypothesis

I expect repeated answers may vary even when the prompt stays similar. A single answer is unlikely to represent every user’s experience.

The Method

  1. I would define buyer-focused prompts, platforms, brand names and a repeat schedule before recording answers.
  2. I would log the exact prompt, time, platform/model where known, account context, location and search mode. Brand mentions, brand-site citations and other sources would be recorded separately.
  3. I would compare repeated observations within the defined sample and retain evidence. I would report variation and missing observations without turning the sample into a market-wide ranking.

The Evidence

I plan to retain dated answer screenshots or excerpts and a complete observation log. No AI searches have been run for this proposal.

The Outcome

Not tested. There is no observed outcome to report.

The Learning

Next investigation: I would decide whether the sampling method is repeatable enough to justify a larger study. I would not infer referral traffic or conversions from answer appearances.

04AI-assisted SEO workflows / Technical SEO

Can AI-assisted URL clustering reduce manual SEO research work without compromising accuracy?

Planned · Proposal
Published status: PlannedProposed experiment · not yet tested

The Question

Can AI-assisted URL clustering reduce manual SEO research work without compromising accuracy?

The Hypothesis

I expect assisted clustering could shorten an initial sorting task, but the review and correction time may reduce or remove that saving.

The Method

  1. I would define clustering rules and prepare an approved URL sample with a manually reviewed reference grouping.
  2. I would run the manual and assisted workflows on comparable tasks, recording setup, processing, review and correction time.
  3. I would review incorrect groups and missed distinctions against the same criteria, record tool/model and prompt details, and repeat with a different sample before generalising.

The Evidence

I plan to retain the approved input set, reference groups, tool output, error log and time records. No timed comparison or accuracy assessment has been completed.

The Outcome

Not tested. There is no observed outcome to report.

The Learning

Next investigation: I would compare total reviewed-workflow time, not generation speed alone, and investigate which page types require manual judgement.

05Conversion and landing pages / Content optimisation

Do alternative landing-page headlines improve clarity and conversion performance?

Planned · Proposal
Published status: PlannedProposed experiment · not yet tested

The Question

Do alternative landing-page headlines improve clarity and conversion performance?

The Hypothesis

I expect a more specific headline could improve understanding, but it may have no effect—or a negative effect—on qualified enquiries.

The Method

  1. I would document the current headline, alternative copy and the specific misunderstanding I want to address.
  2. I would first compare comprehension with a consistent reader task. If a traffic test is appropriate, I would define the allocation, primary outcome, guardrails, sample plan and stopping rule before launch.
  3. I would keep other page elements consistent where possible, validate conversion events and report uncertainty, lead quality and any operational changes.

The Evidence

I plan to retain headline versions, comprehension notes, allocation details and validated event reports if a traffic test proceeds. No variants have been tested and no conversion results exist.

The Outcome

Not tested. There is no observed outcome to report.

The Learning

Next investigation: I would separate clearer communication from a proven conversion effect. Low volume or conflicting evidence may leave the result inconclusive.

Adding a real experiment

Each entry uses the same six-part record. Dates, sample sizes and approved evidence appear only when supplied. Keep negative and inconclusive outcomes visible; explain uncertainty rather than substituting a success claim.

  1. Copy the reusable JSON template and assign a unique ID and number.
  2. Record your question, hypothesis and method before testing.
  3. Add the actual date, sample size and unit when evidence exists. Add only approved evidence with an accurate caption.
  4. Update the published status, outcome and learning from the documented observations, then rebuild the site.
Download the experiment template ↓

Planned: not started. In Progress: work underway. Completed: the planned method is finished, regardless of whether the hypothesis was supported. Inconclusive: evidence cannot resolve the question reliably.

Have a question worth testing?

Discuss an experiment ↗