How to Compare Technical SEO Crawlers on the Same Site

Two crawler exports can disagree without either tool being “wrong.” One may render JavaScript, another may obey a different robots setting, and a third may deduplicate parameters before you ever see them. A fair comparison begins with a site whose expected defects are known and a configuration sheet strict enough to explain every difference.

The buying decision is not “Which crawler finds the most issues?” It is “Which operating model produces evidence our team can trust, explain and act on at our real crawl size?”

Start with the operating model

Do not trial all three when the deployment requirement already eliminates one. A team prohibited from sending crawl data to a managed platform should settle that gate before comparing dashboards.

Choose a site with known answers

The safest fixture is a staging property where you can seed defects. If production is the only option, document a sample manually before the bake-off. Include at least one redirect chain, canonical conflict, noindex URL in the sitemap, blocked resource, duplicate title, orphan page, JavaScript-inserted link, hreflang error and parameterised duplicate.

Write the expected URL and response evidence for every seeded issue. Do not grade a tool on total issue count; a crawler can “win” that contest by producing more low-value warnings.

Freeze the crawl contract

Record starting URLs, allowed hosts/subdomains, robots behaviour, user agent, crawl speed, URL parameter rules, authentication, rendering mode and resource types. Use the same scope in every candidate. Save the configuration and tool version.

This matters particularly across operating models. Screaming Frog and Sitebulb Desktop use local resources; Sitebulb Cloud and Lumar move more of the run into a managed environment. A cloud crawl should not receive a wider scope or different credentials merely because setup is easier.

Establish a server-safety budget

Agree maximum requests per second and prohibited routes with the technical owner. Exclude logout, checkout, destructive actions, infinite calendars and faceted traps. Watch response time and error rate during the run. Stop if the crawler changes the site's behaviour.

Crawler speed is not a useful advantage when it overloads the fixture or receives a different response mix. Compare within the same safe rate first; test maximum throughput separately if it is commercially important.

Compare detection with a confusion matrix

For the seeded set, count:

• true positives: known issues correctly found;
• false negatives: known issues missed;
• false positives: flagged conditions that are not defects under the site's rules;
• unverified findings: plausible issues that need manual investigation.

Keep detection separate from priority. Sitebulb's Hints may reduce explanation work; Screaming Frog's exports and custom extraction may reveal evidence the analyst can shape more precisely; Lumar's managed platform may support recurring enterprise monitoring. Those are workflow differences, not permission to change what “correctly found” means.

Test JavaScript as a separate experiment

Run one crawl without rendering and one with the same rendering assumptions in each eligible product. Compare discovered URLs, canonicals, titles, links and blocked resources. Inspect a sample of discrepancies in the rendered DOM.

Record rendering time, peak memory or platform consumption and any timeout. A crawler that finds more JavaScript links may be correct—or may be executing a state the search engine/user never reaches. Ground truth still decides.

Model local memory before a large crawl

For desktop crawlers, URL count is only a rough planning input. HTML size, rendered resources, JavaScript execution, stored response bodies and extraction fields can change memory use materially. Run a bounded sample first and record peak memory, elapsed time and disk usage.

Project the large run from observations rather than a universal “bytes per URL” claim:

`projected resource use = observed resource use ÷ sampled URLs × expected URLs × safety factor`

Use the projection as a capacity warning, not a guarantee. Templates, media-heavy sections and parameter traps can make the full site unlike the sample. When a desktop crawl approaches the machine's safe limit, compare database storage or a cloud operating model instead of treating a crash as a product-quality result.

Measure the hand-off

Give the outputs to the person who must act: an engineer, client, content lead or product owner. Allow ten minutes and ask them to identify the first three actions, the supporting URLs and the expected outcome. Record clarification questions and analyst preparation time.

This test prevents a common procurement mistake. Raw crawl power is valuable to a specialist; prioritised explanations are valuable to a broader team. The better product is the one that reduces the expensive hand-off in your actual organisation.

Fix two defects and rerun

Repair one template-level problem and one URL-level problem. Re-run with the original configuration. Evaluate whether each product distinguishes new, resolved and persistent issues, and how easily the team can explain the change.

Scheduled monitoring should also be tested across a missed run or authentication failure. An attractive trend chart is not reliable if failures silently create gaps.

Price an accepted audit

Use:

`licence/plan share + crawler infrastructure + configuration hours + analysis hours + reporting hours ÷ accepted audits`

Screaming Frog publishes a free 500-URL boundary and a paid per-user annual licence. Its practical scale depends on machine resources. Sitebulb splits Desktop and Cloud economics. Lumar is quote-based and should be evaluated from a written entitlement schedule. Preserve these models; do not turn them into a single imaginary “cost per URL” unless each contract actually meters URLs the same way.

The Screaming Frog free limit can answer whether a 400-URL site fits a basic local workflow, but it cannot model a 40,000-URL JavaScript crawl. Conversely, a quote-based platform should not be dismissed as “more expensive” until the quote includes recurring monitoring, shared access, history and implementation that the desktop route would require the team to supply.

Scorecard

Reject the comparison if scope, rendering, rate or ground truth changed between runs. Mark a price incomplete when the quote omits sites, users, crawl volume, retention or support. The point is not to manufacture a winner; it is to discover which operating model makes reliable technical evidence cheaper to produce and act on.

Make the next test decisive

Open the Screaming Frog SEO Spider analysis when local flexibility and a specialist operator fit. Open Sitebulb when the hand-off from crawl to prioritized action is the suspected bottleneck. Use Lumar when scheduled enterprise monitoring and shared governance justify a sales-assisted evaluation.

Then run one seeded fixture through only the two viable operating models. Buy when the winning route finds the known defects, explains its false positives, survives the rerun and lowers cost per accepted audit. A larger issue count is not a purchase result.

Official sources used for the example boundaries

• Screaming Frog SEO Spider features and licence pricing ↗ — checked 2026-08-26.
• Screaming Frog licensing and crawl FAQ ↗ — checked 2026-08-26.
• Screaming Frog configuration reference ↗ — checked 2026-08-26.
• Sitebulb Desktop subscription pricing ↗ — checked 2026-08-26.
• Sitebulb Desktop features ↗ — checked 2026-08-26.
• Sitebulb Desktop trial setup ↗ — checked 2026-08-26.
• Lumar platform overview ↗ — checked 2026-08-26.
• Lumar packaging and quote process ↗ — checked 2026-08-26.
• Lumar technical SEO metrics ↗ — checked 2026-08-26.