ML / AI向けのツール

Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.

Document parser from OpenDataLab that turns PDFs, images and Office files into Markdown or JSON, handling tables, formulas and complex layouts well, with page-by-page reading and citation anchors for agents. Parses locally by default; needs Python 3.10+.

Classic open-source OCR engine and CLI that reads text from images and outputs plain text, hOCR, searchable PDF or TSV. Supports 100+ languages; install the language data for each language you need.

All-in-one LLM fine-tuning framework: llamafactory-cli handles pretraining, SFT, PPO/DPO alignment and LoRA/QLoRA training, plus chat inference, weight merging and export, and OpenAI-style API serving for 100+ open models. Has a Web UI; training usually needs a GPU.

Multi-agent orchestration framework for Claude Code and Codex, formerly Claude Flow: swarm coordination, self-learning memory and cross-machine federated communication. The marketplace offers 30+ plugins; full features need npx ruflo init.

Multi-agent orchestration framework formerly called Claude Flow: ruflo init gives Claude Code nearly 100 specialized subagents, swarm coordination, cross-session vector memory and hooks; add --codex to configure Codex instead. Also works as a standalone MCP server or plugin marketplace.

Job search system that runs in your local AI coding CLI: scans job portals, scores roles with a structured A-H evaluation and a 1-5 rating, flags fake listings, tailors ATS resumes and cover letters, and tracks applications. Never submits for you. Needs Node.js.

Parses PDF, DOCX, PPTX, XLSX, HTML, images and audio, recovering layout, reading order, tables and formulas, and exports Markdown, HTML or lossless JSON. Started by IBM Research, now hosted by the LF AI & Data Foundation; runs fully locally.

Command-line toolkit for transcoding, cutting, joining, frame extraction and streaming of audio and video, including ffmpeg, ffprobe and ffplay. A core dependency for agents handling media. The GitHub repo is the official mirror.

Open-source crypto trading framework: download market data, backtest and optimize strategies, and paper trade from the CLI while agents iterate on strategies. Docker recommended; strategies need Python and live trading needs exchange API keys. Backtests don't guarantee returns.

General-purpose AI agent that runs on your machine, with a desktop app, CLI and API. Written in Rust, works with 15+ model providers and connects to 70+ extensions via MCP. Originally from Block, now part of the Linux Foundation's AAIF.

C/C++ port of OpenAI's Whisper speech recognition model that transcribes audio offline and outputs subtitles. Runs on CPU, with Apple Silicon and various GPU acceleration. Download a ggml-format model first.

From HKU's Data Intelligence Lab: plugins or skills for Claude Code, Codex and other agents generate agent-callable CLIs for apps like GIMP, Blender and LibreOffice, and the cli-hub package manager finds and installs ready-made community CLIs. Needs Python 3.10+.

Astral's extremely fast Python linter and formatter written in Rust, replacing Flake8, isort and Black, with 900+ built-in rules and auto-fix.

Terminal multiplexer: run multiple sessions and panes in one terminal, with processes surviving disconnects. Agents use it to host long-running tasks and drive interactive programs, and many parallel multi-agent session tools build on it.

Microsoft's open-source AI quant investment research platform: the qrun command runs dataset building, model training, backtesting and evaluation from a YAML config, with many benchmark models. The official dataset is paused, so prepare market data per the README first.

Lightweight multi-protocol command-line downloader that fetches in parallel segments from multiple sources over HTTP(S), FTP, SFTP, BitTorrent and Metalink. Resumes downloads, reads URLs in bulk from text files and offers a JSON-RPC interface for scripts or agents.

MLOps plugin from the wshobson/agents marketplace: data scientist, ML engineer and MLOps engineer subagents and a /ml-pipeline command that runs the pipeline in phases from data analysis and training to deployment and monitoring, plus ML pipeline and recommender system skills.

Plugin for building LLM apps: LangGraph agents with memory and tool use, hybrid-search RAG, vector index tuning, prompt optimization and LLM evaluation. Includes 3 subagents and 8 skills.

Quant trading plugin from the wshobson/agents marketplace: quant analyst and risk manager subagents plus backtesting and risk metrics skills for strategy backtests, portfolio optimization and risk measures like VaR, Sharpe ratio and max drawdown. Needs local Python; not investment advice.

Converts PDFs, images, PPTX, DOCX, XLSX, EPUB and more into Markdown, JSON or HTML, preserving tables, formulas and layout, with optional LLM help for higher accuracy. Needs Python 3.10+ and PyTorch; large-scale commercial use of the model weights needs a paid license.

Designs and builds MCP servers: first picks a deployment model among remote HTTP, MCPB bundles and local stdio, then guides tool design, auth and in-chat interactive components. Maintained by Anthropic.

Connects Claude Code to the Pyright language server for static type checking and code intelligence in Python. Install pyright first with npm, pip or pipx. Official Anthropic plugin.

Creates or improves Agent Skills from scratch: writes eval cases to test them, runs benchmarks with variance analysis, and tunes descriptions for more accurate triggering. Maintained by Anthropic.