Changelog

What's new on CrawlFox. Newest first. Subscribe via the RSS feed (coming soon).

2026-05-10

Tightened the customer API surface — request responses no longer reveal which extractor or proxy served the result.
Hardened the password-reset flow end-to-end with new /v1/auth/forgot and /v1/auth/reset endpoints.
Pruned marketing copy of unsubstantiated metrics and dead links; refreshed the homepage code samples to match the live gateway contract.
Cookie persistence fix in the Camoufox tier — sessions now stick when the cascade pre-bypasses a cheap fetch.

Smarter routing when a proxy refuses tunneling: we now try the in-house browser tier before falling back to the external extractor.
Tightened the extractor selector so vendor-only modes don't compete on the regular threshold path.

Added a final-resort external extractor rung to the cascade — proxy-blocked failures now escalate cleanly instead of giving up.
Replaced the old length-based escalation heuristic with a vendor-agnostic bot-wall classifier.
Per-attempt detail now records the literal extractor name on the winning rung, so the activity log can show exactly how a request was served.
Admin /logs modal now shows the full escalation ladder and which extractor succeeded.

New customer-facing per-scrape detail page — drill into any request to see attempts, timings, and outcome.
Crash-resistant browser pools (Camoufox + Obscura) with bounded concurrency and recycle-after-N policies.
In-page wait for Cloudflare Managed Challenge — most CF sites now resolve without bouncing through an outer retry.
Gateway refuses to cache bot-wall, error, empty, or challenge bodies — so a poisoned response can't infect later requests.
Mobile polish across the playground, logs, batch jobs, and docs — popovers, tab strips, and tables now stay inside the viewport.

Playground redesign — tabbed multi-format output, runnable curl snippets baked into the keys flow, persistent Recent Runs across reloads.
Self-serve API key rotation from the dashboard.
New 'Main content only' toggle and auto-prepended https:// for URLs without a scheme.
Force-fresh toggle and cache-hit badge so you can tell when a result came from cache.
SERP got Cards + Table views and dropped the Live tab; defaults to raw JSON with cards on toggle.

Dashboard now talks to the gateway for users and batch jobs instead of holding its own user table — single source of truth.
Admin: hot-hosts and strategies pages, realign-sequence button, plus user create/delete UI with toast feedback and spinner buttons.
Async proxy combobox replaces the old full-list <select> — pages with thousands of proxies stay responsive.

Playground gained Links and Images tabs — extracted links and the scraped image gallery without leaving the page.
Preview/Raw toggle for markdown output.
Critical security, performance, and code-quality overhaul across the gateway.

Per-job URL cap raised from 50k to 100k for batch jobs.
Server-action body size limit raised to 25 MB so larger batch submissions don't fail at the edge.

Per-cookie list and detail pages with a cookie-detail timeline in the admin pool view.
Proxy IP-version field added; the Mobile/ISP tier variants were dropped in favour of a cleaner classification.

Admin Logs page — paginated table, filter bar, drill-down modal, error-category and per-endpoint charts.
Admin overview page with live KPIs and a Cookie Pool card.
Admin API Keys page — create, list, enable/disable, see usage.

Premium dashboard redesign — Firecrawl-inspired shell, cards, primitives.
Sticky scroll-aware nav on the marketing site, premium auth redesign.
Progressive SERP rendering via NDJSON stream — results appear as they arrive instead of after the full job completes.

First production-ready dashboard pass — playground, legal pages, logs, charts.
Premium landing page redesign showcasing the full feature set.
Playground and SERP now authenticate to /v1/scrape and /v1/serp with the customer's API key, not an admin token.

Dashboard 'Wave 1' shipped — playground, batch jobs, admin users/plans, proxy upgrades.
Bulk-proxy upload now chunks below the gateway 10k cap, normalises common provider formats, and shows live progress with a concurrency selector.
Premium SaaS layout with role-aware sidebar and light/dark theme.