Tools

In-process analytical SQL database. Its CLI queries CSV, Parquet and JSON files directly with SQL, no import or server needed, so agents can sample and aggregate large files. Officially maintained.

Adaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use.

Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.

Command-line edition of the open-source OpenBB investment research platform: query stock prices, financial statements, macroeconomic and other financial data from the terminal, and automate data pulls with Routine scripts. Officially maintained; some sources need API keys.

Cross-platform shell built around structured data: pipelines pass tables instead of plain text, and you can open and query JSON, CSV, TOML and more directly. Default builds include nu --mcp, which serves it to agents as an MCP server.

All-in-one security scanner for container images, filesystems, Git repos and Kubernetes clusters: finds known vulnerabilities, IaC misconfigurations, leaked secrets and software licenses, and generates SBOMs. Open source, maintained by Aqua Security.

HashiCorp's secrets management system: the vault CLI reads and writes secrets, issues dynamic database and cloud credentials on demand, encrypts and decrypts data, and manages leases and revocation. Needs a reachable Vault server.

Fast vulnerability scanner driven by YAML templates: uses a community template library to detect known vulnerabilities over HTTP, DNS, TCP, SSL and more, supports custom templates and plugs into CI/CD. Use only on targets you're authorized to test.

Turns websites and Electron desktop apps into deterministic CLIs, with built-in adapters for Reddit, Hacker News, Bilibili, Zhihu, Xiaohongshu and more, and lets agents operate any page through your signed-in Chrome. Needs its companion browser extension.

Detects hardcoded passwords, API keys and tokens in Git repos, files and stdin, and can run as a pre-commit hook or GitHub Action to block leaks. Now feature-frozen with security fixes only, as the author has moved on to Betterleaks.

Finds leaked credentials in Git, GitHub, S3, filesystems and more, recognizing 800+ secret types and checking with each service whether a key is still live to cut false positives.

CLI and library for evaluating and red-teaming LLM apps: compare prompts and model outputs in bulk, scan for security vulnerabilities and plug into CI/CD, with evals running locally. Now part of OpenAI and still open source under MIT.

Open-source reverse engineering framework and CLI: disassemble, debug, emulate and patch binaries across many architectures and file formats, scriptable through r2pipe or built-in JavaScript.

Encrypts secret values in YAML, JSON, ENV and INI config files using AWS, GCP or Azure KMS, age or PGP, so secrets can be safely committed to Git alongside code. A CNCF sandbox project.

Alibaba's open-source offline data sync tool: JSON job configs batch-sync between heterogeneous sources like MySQL, Oracle, PostgreSQL, Hive, HDFS and MaxCompute, run with python datax.py. Needs JDK 1.8+ and Python; download and unpack the official toolkit.

Semantic multi-language static analysis: find vulnerabilities and bugs with rules that look like source code, across 30+ languages with custom rules. Scans locally by default without uploading code; cross-file analysis and Pro rules require sign-in.

Security baseline and compliance checks for AWS, Azure, GCP, Kubernetes, GitHub, M365 and more, with built-in CIS, NIST, PCI-DSS, SOC 2 and other frameworks. Generates reports or opens a local dashboard. Configure credentials for each cloud account first.

Command-line framework for warehouse modeling and ELT transformations: write models in SQL and let dbt handle dependencies, data tests and documentation. v2 is rewritten in Rust as a single binary. Maintained by dbt Labs; needs a data warehouse connection.

Classic network discovery and security auditing tool: host discovery, port scanning, service version and OS detection, plus vulnerability checks with NSE scripts. Only scan networks you are authorized to test.

PostgreSQL command-line client with smart autocompletion and syntax highlighting. Compatible with psql environment variables, and -c runs a statement and exits, so agents can query databases non-interactively.

Scans container images, filesystem directories and SBOMs for known vulnerabilities across major Linux distro packages and many language ecosystems, ranks risk by EPSS and KEV, and filters results with OpenVEX. Good for vulnerability gates in CI. Maintained by Anchore.

Simon Willison's LLM CLI and Python library: call OpenAI, Claude, Gemini and local models from the terminal, with conversations logged to SQLite automatically, plus embeddings, structured extraction, tool calling and a plugin system.

MySQL terminal client with smart autocompletion and syntax highlighting, also compatible with MariaDB and TiDB. Supports --execute and batch mode with CSV or table output, so agents can run SQL directly.

Publishes SQLite databases as browsable, queryable web pages and JSON APIs, with SQL queries, metadata and plugins, and deploys to Cloud Run and other platforms with datasette publish. Great for data exploration and data journalism. Needs Python 3.10+.