Competitor Watch & Monthly Digest
Watches each client's competitor websites, detects what changed since last month, scores how much it matters, and writes a digest an account lead can read in 90 seconds.
For a businessGives account teams a 90-second monthly read on what competitors changed, instead of hours spent manually checking websites.
- Status
- Spec build: built to a real brief, run on real competitor pages
- Result
- Not measured against an answer key: run end to end on live pages
- How it was measured
- A fixture backdates stored snapshots, so the monthly comparison and the materiality scoring can be exercised on demand
- Biggest limit
- How well it judges what is material hasn’t been scored yet
01 · Context The problem
Competitor websites change constantly, and almost none of it matters. The job is deciding which change is material — new case results, attorneys, awards or news — and staying quiet about the rest, because a digest full of trivial edits gets ignored and nobody has time to reread nine websites a month.
02 · Pipeline How it works
- Build the registry
Reads the internal links on each competitor homepage and has Claude identify the case-results, attorney, awards and news pages by what they are, whatever the site calls them. Confidence ≥ 0.5, one page per category, capped at 160 links.
- Fetch each month
Each tracked page is fetched and compared with its stored snapshot.
- Detect real changes
Scripts, styles, nav and footers are stripped, then a djb2 hash is compared. Only new sentences over 40 characters move forward. Each page is marked baseline, unchanged, changed or fetch_failed.
- Score materiality
Five calibrated bands from routine (0–0.2) to critical (1.0) across 7 categories. Findings under 0.3 are dropped, and new text must exceed 60 characters to be scored.
- Group per client
Findings at or above 0.65 are grouped into each client's digest.
- Write the brief
Readable in 90 seconds, never mentions AI, invents nothing, and ends with "What we recommend." A fallback message is used if the call fails, and each client gets a ready-to-send flag.
03 · Choices Design decisions
Calibration lives in the prompt
The scoring prompt says: if most items score above 0.6, you are miscalibrated.
The run watches itself
It reports score spread and a signal ratio (material findings vs changes detected) to catch a threshold that has drifted too low.
Failures are never dropped
A scoring failure becomes a REVIEW_MANUALLY item at materiality 1, so it lands at the top instead of disappearing.
Testable without waiting a month
A fixture trims stored snapshots, recomputes their hashes and backdates them 30 days, so the next run sees real new content.
04 · Evidence How I tested it
Run against 17 tracked pages across 9 competitor sites, using the backdating fixture to exercise the diff and materiality scoring end to end.
05 · Honest notes Known limits
- Snapshots are capped at 20,000 characters per page.
- Pages with under 500 characters of HTML are skipped when building the registry.
Want to see this one run?
I'll screen-share the workflow, the test set and the results, including what didn't work.