Highlight Claude Code MCP integration prominently
- Added Claude Code MCP callout in opening section - Featured 'Claude Code Ready' in features list - Added dedicated MCP integration example after Quick Start - Positions Crawailer specifically for AI agent and MCP server use cases
This commit is contained in:
parent
8b0fa8ef77
commit
2e1e7d49eb
24
README.md
24
README.md
@ -4,6 +4,8 @@
|
||||
|
||||
> **Finally!** A Python library that handles React, Vue, Angular, and dynamic content without the headaches. When `requests` fails and Selenium feels like overkill, Crawailer delivers clean, AI-ready content extraction with bulletproof JavaScript execution.
|
||||
|
||||
> ⚡ **Perfect for Claude Code MCP servers** - Built specifically for AI agents and MCP integration
|
||||
|
||||
```python
|
||||
pip install crawailer
|
||||
```
|
||||
@ -16,7 +18,7 @@ pip install crawailer
|
||||
- **🎯 JavaScript-First**: Executes real JavaScript on React, Vue, Angular sites (unlike `requests`)
|
||||
- **⚡ Lightning Fast**: 5-10x faster HTML processing with C-based selectolax
|
||||
- **🤖 AI-Optimized**: Clean markdown output perfect for LLM training and RAG
|
||||
- **🔧 Three Ways to Use**: Library, CLI tool, or MCP server - your choice
|
||||
- **🔧 Claude Code Ready**: Drop-in MCP server integration for AI agents
|
||||
- **📦 Zero Config**: Works immediately with sensible defaults
|
||||
- **🧪 Battle-Tested**: 18 comprehensive test suites with 70+ real-world scenarios
|
||||
- **🎨 Developer Joy**: Rich terminal output, helpful errors, progress tracking
|
||||
@ -60,6 +62,26 @@ research = await web.discover(
|
||||
# Crawailer → Full content + dynamic data
|
||||
```
|
||||
|
||||
### 🧠 Claude Code MCP Integration
|
||||
|
||||
```python
|
||||
# Add to your Claude Code MCP server
|
||||
from crawailer.mcp import create_mcp_server
|
||||
|
||||
@mcp_tool("web_extract")
|
||||
async def extract_content(url: str, script: str = ""):
|
||||
"""Extract content from any website with optional JavaScript execution"""
|
||||
content = await web.get(url, script=script)
|
||||
return {
|
||||
"title": content.title,
|
||||
"markdown": content.markdown,
|
||||
"script_result": content.script_result,
|
||||
"word_count": content.word_count
|
||||
}
|
||||
|
||||
# Now Claude can extract content from ANY website, including SPAs!
|
||||
```
|
||||
|
||||
## 🎯 Design Philosophy
|
||||
|
||||
### For Robots, By Humans
|
||||
|
Loading…
x
Reference in New Issue
Block a user