Best Crawling AI Skills & MCP Servers
8 curated Crawling skills and MCP servers — install any of them into Claude, Cursor, ChatGPT, n8n, or any AI stack with one command.
Alterlab
MCP server for web scraping, structured data extraction, and screenshots — give Claude, Cursor, Windsurf, and AI agents the ability to scrape any website with automatic anti-bot bypass
Novada Search
MCP server for web search, extraction, crawling, and research — powered by Novada's 100M+ proxy IPs across 195 countries. Works with Claude, Cursor, VS Code, and any MCP client.
Webcontext Ai
Turn any web content into clean AI-ready context — with crawling, chunking, semantic search, and MCP tools
Searxng
SearXNG MCP Server with Better Search - 100% privacy-focused intelligent search with auto Docker management and User Agent rotation
Web Docs
MCP server for crawling and indexing web documentation - works with any website
Crawlforge
CrawlForge MCP Server - Professional Model Context Protocol server with 23 web scraping, crawling, and content processing tools. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. v4.0 adds Markdown-firs
exa-mcp
Search the web and codebases to get precise, up-to-date context for programming and research. Find examples, API usage, and documentation from real repositories and sites to ship faster with fewer mistakes. Extend investigations with deep search, crawling, and business or profile
Exa Search
Fast, intelligent web search and web crawling. Get fresh information about libraries, APIs, and SDKs.
About Crawling skills on iClaude
iClaude is the universal install layer for AI skills. Every Crawling skill on this page can be installed into Claude Code, Claude Desktop, Cursor, ChatGPT, n8n, Codex, and more — using a single copy-paste command. No config drift, no per-stack adapters, no manual MCP wiring.