Инструменты

Open-source browser agent project. Its browser-use CLI lets Claude Code, Codex and other agents drive your logged-in local Chrome with Python snippets, or connect to a cloud browser; also ships a Python agent library. The cloud browser needs an API key.

Official MCP reference server that fetches web pages and converts them to Markdown for the model, reading long pages in chunks from a given offset. Zero config, but it can reach internal network addresses, so use it with care.

MCP server built into the adaptive Python scraping framework Scrapling: supports plain HTTP requests, browser rendering and stealth fetching, and uses CSS selectors to pass only the target content to the AI, saving tokens and filtering out prompt injection.

Adaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use.

Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.

Job search system that runs in your local AI coding CLI: scans job portals, scores roles with a structured A-H evaluation and a 1-5 rating, flags fake listings, tailors ATS resumes and cover letters, and tracks applications. Never submits for you. Needs Node.js.

Browser automation MCP server open-sourced by ByteDance in the UI-TARS-desktop repo: drives pages through Puppeteer accessibility data with an optional vision mode, running locally over stdio or remotely over SSE or Streamable HTTP.

Headless browser written from scratch in Zig for AI agents and automation, using less memory and running faster than headless Chrome in official benchmarks. fetch turns pages into Markdown, and it can serve CDP or MCP. Telemetry is on by default (an env var disables it); Windows needs WSL.

Bundles a whole web page with its CSS, images, fonts and scripts into a single HTML file that opens offline exactly as it looked, good for archiving pages or handing them to an agent for offline analysis. It does not run JavaScript, so pre-render dynamic pages with a headless browser.

Turns 18 kinds of sources, including doc sites, GitHub repos, PDFs and videos, into skill packages for Claude and other platforms, or exports RAG chunks and vector store data. Optional MCP server, and it can call a local agent like Claude Code to enrich content. Needs Python 3.10+.

Extracts the main content of a web page, stripping comments, sidebars, headers and footers, and outputs clean HTML or Markdown with metadata like title, author and publish date, saving tokens when agents read the web. Built for Obsidian Web Clipper by kepano; still in development.

Python library and CLI for extracting main web content: fetches pages, crawls via sitemaps and RSS feeds, strips headers, footers and other noise, and outputs TXT, Markdown, JSON, XML and more with metadata. Also processes downloaded HTML offline.

One command extracts or summarizes web pages, PDFs, YouTube videos, podcasts and media files, with JSON output. Summaries can use many model APIs, local models or a logged-in Codex or Claude CLI; plain extraction needs no key. Also has a browser sidebar extension.

Gives agents Exa web search and page content fetching, with an optional multi-step research agent and advanced filtered search. Maintained by Exa; the hosted service works anonymously with limits, and signing in or adding an API key raises them.

Converts PDF, Word, Excel, PowerPoint, images, audio, web pages and YouTube transcripts into Markdown that agents can read and cite. Built on Microsoft MarkItDown; the Docker image lacks audio transcription and image recognition, so build locally for full features.

Four tools for real-time web search, page content extraction, site mapping and full-site crawling, available as a remote server or run locally. Maintained by Tavily; needs an API key or OAuth sign-in, and a free account is available.

Multi-engine web search MCP with no API key, supporting Bing, DuckDuckGo, Brave, Baidu, Sogou, CSDN, Juejin and more; also fetches page content, CSDN articles and GitHub READMEs. Ships a CLI and a local server too.

Web search through DuckDuckGo plus fetching and parsing page content, with built-in rate limiting and LLM-friendly result formatting. No API key needed; runs with a single uvx command.

A plugin marketplace with dozens of plugins: an enhanced skill-creator, scrapers for Bilibili, Douban and Feishu docs, China A-share news and US stock fundamentals, WeCom notifications, deep research and fact-checking, with docs in English and Chinese.

Web search and page reading for agents through a self-hosted or trusted SearXNG metasearch instance, with no paid search API key. Requires a SearXNG instance with JSON output enabled.

Jina AI's official remote MCP: converts web pages and PDFs to Markdown, searches the web, arXiv, SSRN and images, and offers screenshots, reranking and semantic dedup. Reading tools work without a key (rate-limited); search and rerank need a Jina API key.

Official Firecrawl skill: drives the Firecrawl CLI to search, scrape, map site URLs, batch-crawl, interact with pages and monitor changes, returning LLM-ready Markdown. Search and scrape have a rate-limited free tier; crawling and monitoring need an account or self-hosted instance.

Firecrawl's official CLI: scrape, search, map and crawl sites and interact with pages in a browser from the terminal, with Markdown output; can also install companion skills for coding agents. Needs a Firecrawl account or API key, or point it at a self-hosted instance.

Turns web pages, long PDFs, Word files, images or code into tutor-style study notes: Markdown notes plus searchable HTML knowledge cards with concept breakdowns and mind maps. Needs Python 3.8+, plus Chrome for some sites; the optional tutor chat needs an OpenAI-compatible API key.