Best Crawl AI Skills & MCP Servers

MCP Server for Driflyte

The web data platform for AI agents — fetch, search, crawl, extract, monitor, screenshot. 55+ domain extractors, 65-98% token savings, MCP server included. One API call.

MCP server for web scraping, structured data extraction, and screenshots — give Claude, Cursor, Windsurf, and AI agents the ability to scrape any website with automatic anti-bot bypass

MCP server for web search, extraction, crawling, and research — powered by Novada's 100M+ proxy IPs across 195 countries. Works with Claude, Cursor, VS Code, and any MCP client.

MCP server for PubMed literature search and US/UK drug labelling — built by PharmaTools.AI

MCP server for PubMed literature search and US/UK drug labelling — built by PharmaTools.AI

Non-custodial x402 MCP server for AI agents. Lets Claude, Cursor, LangChain and CrewAI pay for HTTP 402–gated APIs with USDC micropayments on Base L2. Unlike Cloudflare Pay-Per-Crawl, works on any host and settles agent-to-publisher direct.

Naver Finance web crawler with MCP server support - crawl Korean stock market data

Smart MCP tool to find and validate movie/tv-show resources with multiple sources support

Turn any web content into clean AI-ready context — with crawling, chunking, semantic search, and MCP tools

MCP server for web scraping — extract clean markdown, links, and metadata from any URL. Zero auth. Free alternative to Firecrawl.

SearXNG MCP Server with Better Search - 100% privacy-focused intelligent search with auto Docker management and User Agent rotation

Agent-optimized MCP server for extracting clean, structured content from web pages - built specifically for AI agents that need LLM-friendly web data

MCP server for crawling and indexing web documentation - works with any website

CrawlForge MCP Server - Professional Model Context Protocol server with 23 web scraping, crawling, and content processing tools. Defaults to local Ollama for LLM extraction (no API key needed); OpenAI/Anthropic available as opt-in. v4.0 adds Markdown-firs

One MCP server for all web data. Search, scrape, crawl, proxy, and AI research in one install.

MCP server for advanced web search using Tavily

Playwright-based same-origin documentation crawler for docs-to-mcp

MCP server for integrating Omnisearch with LLMs

Transparent API key rotation proxy for MCP search providers — Exa, Firecrawl, Tavily, Linkup, Bright Data, Olostep, Dappier, Parallel

AHTML semantic snapshot schema — types, validator, builder, and dual-format serializers (JSON + token-optimal compact text). The agent-readable contract layer for the web.

MCP server for Firecrawl — search, scrape, and interact with the web. Supports both cloud and self-hosted instances. Features include web search, scraping, page interaction, batch processing, and LLM-powered content analysis.

Fast, intelligent web search and web crawling. Get fresh information about libraries, APIs, and SDKs.

Search the web and codebases to get precise, up-to-date context for programming and research. Find examples, API usage, and documentation from real repositories and sites to ship faster with fewer mistakes. Extend investigations with deep search, crawling, and business or profile