도구

Open-source browser agent project. Its browser-use CLI lets Claude Code, Codex and other agents drive your logged-in local Chrome with Python snippets, or connect to a cloud browser; also ships a Python agent library. The cloud browser needs an API key.

MCP server built into the adaptive Python scraping framework Scrapling: supports plain HTTP requests, browser rendering and stealth fetching, and uses CSS selectors to pass only the target content to the AI, saving tokens and filtering out prompt injection.

Adaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use.

Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.

Job search system that runs in your local AI coding CLI: scans job portals, scores roles with a structured A-H evaluation and a 1-5 rating, flags fake listings, tailors ATS resumes and cover letters, and tracks applications. Never submits for you. Needs Node.js.

Browser automation MCP server open-sourced by ByteDance in the UI-TARS-desktop repo: drives pages through Puppeteer accessibility data with an optional vision mode, running locally over stdio or remotely over SSE or Streamable HTTP.

Headless browser written from scratch in Zig for AI agents and automation, using less memory and running faster than headless Chrome in official benchmarks. fetch turns pages into Markdown, and it can serve CDP or MCP. Telemetry is on by default (an env var disables it); Windows needs WSL.

Turns websites and Electron desktop apps into deterministic CLIs, with built-in adapters for Reddit, Hacker News, Bilibili, Zhihu, Xiaohongshu and more, and lets agents operate any page through your signed-in Chrome. Needs its companion browser extension.

Skyvern's official MCP: drives a browser with vision LLMs to complete multi-step tasks like filling forms, downloading files and web research. Connect to Skyvern Cloud (OAuth or API key), or self-host Skyvern and run it as a local process.

Turns 18 kinds of sources, including doc sites, GitHub repos, PDFs and videos, into skill packages for Claude and other platforms, or exports RAG chunks and vector store data. Optional MCP server, and it can call a local agent like Claude Code to enrich content. Needs Python 3.10+.

A Chrome extension that hands the browser you use every day to your agent, keeping your logins and settings, with screenshots, network monitoring, bookmarks and history, and semantic search over tab content. Install the extension and mcp-chrome-bridge first.

Lets agents call thousands of ready-made scrapers and automation Actors from Apify Store to pull data from social media, search engines, maps, e-commerce sites and more. Maintained by Apify; needs an Apify account (OAuth) or API token.

Python library and CLI for extracting main web content: fetches pages, crawls via sitemaps and RSS feeds, strips headers, footers and other noise, and outputs TXT, Markdown, JSON, XML and more with metadata. Also processes downloaded HTML offline.

ScrapeCreators' official skills: mine public TikTok, Instagram, YouTube, Reddit and X data for viral content, customer voice in comments, competitor accounts and ad libraries, and creator vetting, with sourced reports. Needs a ScrapeCreators key (100 free credits, then pay as you go).

Bright Data's official MCP: web search (Google, Bing, Yandex), any page to Markdown, structured data extraction from e-commerce, social and other platforms, and remote browser control. Needs an API token; includes 5,000 free requests a month.

Apify's official skill collection: picks from 130+ curated Actors to scrape public data from social networks, maps, e-commerce sites and more, and can also build, package and deploy Actors. Needs an Apify account; a free tier is available.

Four tools for real-time web search, page content extraction, site mapping and full-site crawling, available as a remote server or run locally. Maintained by Tavily; needs an API key or OAuth sign-in, and a free account is available.

Jina AI's official remote MCP: converts web pages and PDFs to Markdown, searches the web, arXiv, SSRN and images, and offers screenshots, reranking and semantic dedup. Reading tools work without a key (rate-limited); search and rerank need a Jina API key.

Official Firecrawl skill: drives the Firecrawl CLI to search, scrape, map site URLs, batch-crawl, interact with pages and monitor changes, returning LLM-ready Markdown. Search and scrape have a rate-limited free tier; crawling and monitoring need an account or self-hosted instance.

Firecrawl's official CLI: scrape, search, map and crawl sites and interact with pages in a browser from the terminal, with Markdown output; can also install companion skills for coding agents. Needs a Firecrawl account or API key, or point it at a self-hosted instance.

Fetches captions and transcripts for YouTube videos and Shorts, with language selection and fallback, optional timestamps and sponsor segments filtered by default. No API key for transcripts; the optional visual analysis tool needs a TwelveLabs API key and a direct video URL.

Official Tavily skills: let agents search the web, extract page content, discover site URLs, crawl in bulk and run cited deep research through the Tavily CLI (tvly). Install the CLI first; search and extract have a limited no-login tier, other features need sign-in or an API key.

Firecrawl's official plugin: brings the Firecrawl CLI in as skills so Claude Code can search the web, scrape pages to Markdown, map site links or crawl whole sites. Install firecrawl-cli and set an API key first; self-hosting supported.

Official Firecrawl skill: crawls a site and its key pages, checks titles, descriptions, heading structure, internal links and keyword opportunities, compares top-ranking competitors, and outputs a prioritized SEO audit. Needs a Firecrawl API key (free: 1,000 credits/month) and its CLI.