# Tools > Every listed resource, editors’ picks first. The filters below can be combined. - Search: append `?q=` to the URL. Separate several keywords with spaces; all of them must match. Matches name, repository, tags, professions, description and README. - Filter by type: append `?type=` to the URL. - Filter by profession: append `?profession=` to the URL. Separate several ids with commas; a resource matches when it suits any of them. Ids are listed at https://hebbmind.com/api/v1/professions - Filter by tag: append `?tag=` to the URL. Separate several tags with commas; a resource must carry all of them. Tags are listed at https://hebbmind.com/api/v1/tags Resources for DevOps / SRE or ML / AI or Data analysis, Tagged `web-scraping`: 24 items. > - DevOps / SRE: Deployment, monitoring, cloud resources and reliability. > - ML / AI: Model training, evaluation and LLM application development. > - Data analysis: Querying, statistical analysis, reporting and visualization. - [Browser Use](https://hebbmind.com/cli/browser-use.md): [CLI] Open-source browser agent project. Its browser-use CLI lets Claude Code, Codex and other agents drive your logged-in local Chrome with Python snippets, or connect to a cloud browser; also ships a Python agent library. The cloud browser needs an API key. - [Browser Use](https://hebbmind.com/skill/browser-use.md): [Skill] Controls your local Chrome over CDP to navigate, click, type, take screenshots and scrape data, reusing logged-in sessions. Official browser-use skill; install the browser-use CLI with uv first. - [Browser Use MCP](https://hebbmind.com/mcp/browser-use-mcp.md): [MCP] Local MCP server built into Browser Use: lets agents navigate, click, type and extract page content, or hand multi-step tasks to the built-in browser agent. Bring your own OpenAI or Anthropic API key. - [Scrapling](https://hebbmind.com/mcp/scrapling.md): [MCP] MCP server built into the adaptive Python scraping framework Scrapling: supports plain HTTP requests, browser rendering and stealth fetching, and uses CSS selectors to pass only the target content to the AI, saving tokens and filtering out prompt injection. - [Scrapling](https://hebbmind.com/cli/scrapling.md): [CLI] Adaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use. - [Crawl4AI](https://hebbmind.com/cli/crawl4ai.md): [CLI] Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host. - [career-ops](https://hebbmind.com/plugin/career-ops.md): [Plugin] Job search system that runs in your local AI coding CLI: scans job portals, scores roles with a structured A-H evaluation and a 1-5 rating, flags fake listings, tailors ATS resumes and cover letters, and tracks applications. Never submits for you. Needs Node.js. - [@agent-infra/mcp-server-browser](https://hebbmind.com/mcp/browser.md): [MCP] Browser automation MCP server open-sourced by ByteDance in the UI-TARS-desktop repo: drives pages through Puppeteer accessibility data with an optional vision mode, running locally over stdio or remotely over SSE or Streamable HTTP. - [Lightpanda Browser](https://hebbmind.com/cli/browser.md): [CLI] Headless browser written from scratch in Zig for AI agents and automation, using less memory and running faster than headless Chrome in official benchmarks. fetch turns pages into Markdown, and it can serve CDP or MCP. Telemetry is on by default (an env var disables it); Windows needs WSL. - [OpenCLI](https://hebbmind.com/cli/opencli.md): [CLI] Turns websites and Electron desktop apps into deterministic CLIs, with built-in adapters for Reddit, Hacker News, Bilibili, Zhihu, Xiaohongshu and more, and lets agents operate any page through your signed-in Chrome. Needs its companion browser extension. - [Skill Seekers](https://hebbmind.com/cli/skill-seekers.md): [CLI] Turns 18 kinds of sources, including doc sites, GitHub repos, PDFs and videos, into skill packages for Claude and other platforms, or exports RAG chunks and vector store data. Optional MCP server, and it can call a local agent like Claude Code to enrich content. Needs Python 3.10+. - [Trafilatura](https://hebbmind.com/cli/trafilatura.md): [CLI] Python library and CLI for extracting main web content: fetches pages, crawls via sitemaps and RSS feeds, strips headers, footers and other noise, and outputs TXT, Markdown, JSON, XML and more with metadata. Also processes downloaded HTML offline. - [Browserbase Skills](https://hebbmind.com/skill/browserbase-skills.md): [Skill] Browserbase's official skills: drive a local or cloud browser with the browse CLI for web scraping and search, turning site traffic into an OpenAPI spec, adversarial UI testing and competitor research. Cloud features need a Browserbase account. - [Bright Data MCP](https://hebbmind.com/mcp/brightdata-mcp.md): [MCP] Bright Data's official MCP: web search (Google, Bing, Yandex), any page to Markdown, structured data extraction from e-commerce, social and other platforms, and remote browser control. Needs an API token; includes 5,000 free requests a month. - [shot-scraper](https://hebbmind.com/cli/shot-scraper.md): [CLI] Simon Willison's website screenshot CLI: batch screenshots from YAML, record demo videos, export PDF and HTML, run JavaScript in pages to extract data, and dump accessibility trees. Built on Playwright; install a browser on first use. - [Apify Agent Skills](https://hebbmind.com/skill/apify-agent-skills.md): [Skill] Apify's official skill collection: picks from 130+ curated Actors to scrape public data from social networks, maps, e-commerce sites and more, and can also build, package and deploy Actors. Needs an Apify account; a free tier is available. - [Tavily MCP Server](https://hebbmind.com/mcp/tavily-mcp.md): [MCP] Four tools for real-time web search, page content extraction, site mapping and full-site crawling, available as a remote server or run locally. Maintained by Tavily; needs an API key or OAuth sign-in, and a free account is available. - [daymade Claude Code Skills](https://hebbmind.com/plugin/claude-code-skills.md): [Plugin] A plugin marketplace with dozens of plugins: an enhanced skill-creator, scrapers for Bilibili, Douban and Feishu docs, China A-share news and US stock fundamentals, WeCom notifications, deep research and fact-checking, with docs in English and Chinese. - [SearXNG MCP Server](https://hebbmind.com/mcp/mcp-searxng.md): [MCP] Web search and page reading for agents through a self-hosted or trusted SearXNG metasearch instance, with no paid search API key. Requires a SearXNG instance with JSON output enabled. - [Jina AI MCP Server](https://hebbmind.com/mcp/jina-ai-mcp.md): [MCP] Jina AI's official remote MCP: converts web pages and PDFs to Markdown, searches the web, arXiv, SSRN and images, and offers screenshots, reranking and semantic dedup. Reading tools work without a key (rate-limited); search and rerank need a Jina API key. - [Firecrawl](https://hebbmind.com/skill/firecrawl.md): [Skill] Official Firecrawl skill: drives the Firecrawl CLI to search, scrape, map site URLs, batch-crawl, interact with pages and monitor changes, returning LLM-ready Markdown. Search and scrape have a rate-limited free tier; crawling and monitoring need an account or self-hosted instance. - [Firecrawl CLI](https://hebbmind.com/cli/firecrawl-cli.md): [CLI] Firecrawl's official CLI: scrape, search, map and crawl sites and interact with pages in a browser from the terminal, with Markdown output; can also install companion skills for coding agents. Needs a Firecrawl account or API key, or point it at a self-hosted instance. - [Tavily Agent Skills](https://hebbmind.com/skill/tavily-ai-skills.md): [Skill] Official Tavily skills: let agents search the web, extract page content, discover site URLs, crawl in bulk and run cited deep research through the Tavily CLI (tvly). Install the CLI first; search and extract have a limited no-login tier, other features need sign-in or an API key. - [Firecrawl Plugin for Claude Code](https://hebbmind.com/plugin/firecrawl-claude-plugin.md): [Plugin] Firecrawl's official plugin: brings the Firecrawl CLI in as skills so Claude Code can search the web, scrape pages to Markdown, map site links or crawl whole sites. Install firecrawl-cli and set an API key first; self-hosting supported.