Skip to content

Commit 4bf1779

Browse files
committed
feat: add version 0.8.5 release highlights including anti-bot detection, shadow DOM support, and critical security fixes to README
1 parent bb6406a commit 4bf1779

1 file changed

Lines changed: 42 additions & 0 deletions

File tree

README.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -565,6 +565,48 @@ async def test_news_crawl():
565565
## ✨ Recent Updates
566566

567567
<details open>
568+
<summary><strong>Version 0.8.5 Release Highlights - Anti-Bot Detection, Shadow DOM & 60+ Bug Fixes</strong></summary>
569+
570+
Our biggest release since v0.8.0. Anti-bot detection with proxy escalation, Shadow DOM flattening, deep crawl cancellation, and over 60 bug fixes.
571+
572+
- **🛡️ Anti-Bot Detection & Proxy Escalation**:
573+
- 3-tier detection: known vendors, generic block indicators, structural integrity checks
574+
- Automatic retry with proxy chain and fallback fetch function
575+
```python
576+
from crawl4ai import CrawlerRunConfig
577+
from crawl4ai.async_configs import ProxyConfig
578+
579+
config = CrawlerRunConfig(
580+
proxy_config=[ProxyConfig.DIRECT, ProxyConfig(server="http://my-proxy:8080")],
581+
max_retries=2,
582+
fallback_fetch_function=my_web_unlocker,
583+
)
584+
```
585+
586+
- **🌑 Shadow DOM Flattening**:
587+
- Extract content hidden inside shadow DOM components
588+
```python
589+
config = CrawlerRunConfig(flatten_shadow_dom=True)
590+
```
591+
592+
- **🛑 Deep Crawl Cancellation**:
593+
- Stop long crawls gracefully with `cancel()` or `should_cancel` callback
594+
- Works with BFS, DFS, and BestFirst strategies
595+
596+
- **⚙️ Config Defaults API**:
597+
- `set_defaults()` / `get_defaults()` / `reset_defaults()` on BrowserConfig and CrawlerRunConfig
598+
599+
- **🔒 Critical Security Fixes**:
600+
- RCE via deserialization in Docker `/crawl` endpoint — removed `eval()`, added allowlist
601+
- Redis CVE-2025-49844 (CVSS 10.0) — upgraded to 7.2.7
602+
603+
- **60+ Bug Fixes** across browser management, proxy, deep crawling, extraction, CLI, and Docker
604+
605+
[Full v0.8.5 Release Notes →](https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.8.5.md)
606+
607+
</details>
608+
609+
<details>
568610
<summary><strong>Version 0.8.0 Release Highlights - Crash Recovery & Prefetch Mode</strong></summary>
569611

570612
This release introduces crash recovery for deep crawls, a new prefetch mode for fast URL discovery, and critical security fixes for Docker deployments.

0 commit comments

Comments
 (0)