ツール
掲載中のすべての Skill、MCP サーバー、CLI、プラグインです。種類・職業・タグで絞り込むか、キーワードで検索できます。
職業:Data engineering
タグ:python +1
4 件

Scrapling
CLIAdaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use.
D4Vinci/Scrapling105k
Scrapling
MCPMCP server built into the adaptive Python scraping framework Scrapling: supports plain HTTP requests, browser rendering and stealth fetching, and uses CSS selectors to pass only the target content to the AI, saving tokens and filtering out prompt injection.
D4Vinci/Scrapling105k
Crawl4AI
CLIOpen-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.
unclecode/crawl4ai87.6k
Trafilatura
CLIPython library and CLI for extracting main web content: fetches pages, crawls via sitemaps and RSS feeds, strips headers, footers and other noise, and outputs TXT, Markdown, JSON, XML and more with metadata. Also processes downloaded HTML offline.
adbar/trafilatura7.2k