FIELD NOTES / EDITORIAL DRAFT
How I Use Screaming Frog and AI to Review Website Content.
A draft expansion of the workflow already described in my Outcraft project account: AI supports grouping and investigation; the underlying data remains the reference.
Start with the supplied workflow
In my Outcraft account, I describe using Claude with Screaming Frog and Search Console data to group URLs by intent and flag potential content overlap. I review suggestions against the underlying data before using them in recommendations. This article develops that account into a review framework; it is not a timed workflow study.
Keep the inputs visible
The SEO Spider configuration guide includes page titles, headings and indexability among its extraction options. Crawl settings affect what is collected. A review should record its scope and settings, then retain the source export so that an AI-generated grouping can be checked against actual pages.
Reference: Screaming Frog: SEO Spider configuration guide.
Treat a cluster as a suggestion
I would review whether grouped pages really serve the same intent, whether important distinctions were missed and whether a label oversimplifies the content. Similar titles alone do not justify removing pages. The recommendation should explain its evidence and uncertainty, rather than inherit the confidence of the generated summary.
Measure the reviewed workflow
A meaningful efficiency test would include preparation, checking and correction time. It would also assess accuracy using a consistent reference. No time saving or accuracy figure has been established here. An approved sample, review rubric and error log would be needed before publishing those claims.
Source references
- Screaming Frog: SEO Spider configuration guideChecked 2026-10-24