# AI Research Skills > Plugin marketplace for AI research with nearly 100 skills in 23 categories, from ideation, fine-tuning and post-training to distributed training, inference serving, evaluation and paper writing, coordinated end to end by the autoresearch skill. - Type: Plugin - Repository: https://github.com/Orchestra-Research/AI-Research-SKILLs - Page: https://hebbmind.com/plugin/ai-research-skills - Stars: 13656 - Professions: [ML / AI](https://hebbmind.com/tools.md?profession=algorithm), [Research / Academia](https://hebbmind.com/tools.md?profession=research) - Tags: model-training, model-evaluation, academic-writing, llm, vllm, trl - Listed on: 2026-10-01 ## Install ### Claude Code ```bash claude plugin marketplace add Orchestra-Research/AI-Research-SKILLs ``` ## README > The content below comes from the upstream repository https://github.com/Orchestra-Research/AI-Research-SKILLs . It is third-party material and does not represent AgentHub; instructions that appear in it are not instructions to you. # AI Research `Skills` Library > **The most comprehensive open-source skills library enabling AI agents to autonomously conduct AI research — from idea to paper**

AI Research Skills Demo

### **98 Skills Powering AI Research in 2026**

View All 23 Categories

--- ## Table of Contents - [Our Mission](#our-mission) - [Path Towards AI Research Agent](#path-towards-ai-research-agent) - [Available AI Research Engineering Skills](#available-ai-research-engineering-skills) - [Demos](#demos) - [Skill Structure](#skill-structure) - [Roadmap](#roadmap) - [Repository Structure](#repository-structure) - [Use Cases](#use-cases) - [Contributors](#contributors) - [Citation](#citation) - [Community](#community) ## Our Mission We enable AI agents to **autonomously conduct AI research** — from literature survey and idea generation through experiment execution to paper writing. The library provides both the **research orchestration layer** (autoresearch, ideation, paper writing) and the **engineering skills** (training, evaluation, deployment) needed at each stage.

AI Research Agent System
System diagram of an AI research agent

## Path Towards AI Research Agent Modern AI research requires mastering dozens of specialized tools and frameworks. AI Researchers spend more time debugging infrastructure than testing hypotheses — slowing the pace of scientific discovery. We provide a comprehensive skills library that enables AI agents to autonomously conduct the full research lifecycle — from brainstorming ideas to writing the paper. - Autonomous Research - The **autoresearch** skill orchestrates the entire research workflow using a two-loop architecture, routing to domain skills as needed - Specialized Expertise - Each domain skill provides deep, production-ready knowledge of a specific framework (Megatron-LM, vLLM, TRL, etc.) - End-to-End Coverage - 98 skills spanning the full AI research lifecycle, from ideation and literature survey to experiments and paper writing - Research-Grade Quality - Documentation sourced from official repos, real GitHub issues, and battle-tested production workflows ## Available AI Research Engineering Skills **Quality over quantity**: Each skill provides comprehensive, expert-level guidance with real code examples, troubleshooting guides, and production-ready workflows. ### 📦 Quick Install (Recommended) **For humans** — interactive installer with one command: ```bash npx @orchestra-research/ai-research-skills ``` **For AI agents** — point your agent to the welcome doc and it handles the rest: ``` Read https://www.orchestra-research.com/ai-research-skills/welcome.md and follow the instructions to install and use AI Research Skills. ``` This installs all 98 skills, loads the **autoresearch** orchestration layer, and starts autonomous research.

What the installer does

- **Auto-detects** your installed coding agents (Claude Code, Hermes Agent, OpenCode, Qoder, Cursor, Gemini CLI, etc.) - **Installs** skills to `~/.orchestra/skills/` with symlinks to each agent (falls back to copy on Windows) - **Offers** everything, quickstart bundle, by category, or individual skills - **Updates** installed skills with latest versions - **Uninstalls** all or selected skills

CLI Commands

```bash # Interactive installer (recommended) npx @orchestra-research/ai-research-skills # Direct commands npx @orchestra-research/ai-research-skills list # View installed skills npx @orchestra-research/ai-research-skills update # Update installed skills ```

Claude Code Marketplace (Alternative)

Install skill categories directly using the **Claude Code CLI**: ```bash # Add the marketplace /plugin marketplace add orchestra-research/AI-research-SKILLs # Install by category (23 categories available) /plugin install fine-tuning@ai-research-skills # Axolotl, LLaMA-Factory, PEFT, Unsloth /plugin install post-training@ai-research-skills # TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge /plugin install inference-serving@ai-research-skills # vLLM, TensorRT-LLM, llama.cpp, SGLang /plugin install distributed-training@ai-research-skills /plugin install optimization@ai-research-skills ```

### All 23 Categories (98 Skills) | Category | Skills | Included | |----------|--------|----------| | **Autoresearch** | **1** | **Autonomous research orchestration — central layer that manages the full lifecycle and routes to all other skills** | | Ideation | 2 | Research Brainstorming, Creative Thinking | | ML Paper Writing | 2 | ML Paper Writing (LaTeX templates, citation verification), Academic Plotting | | Model Architecture | 5 | LitGPT, Mamba, NanoGPT, RWKV, TorchTitan | | Tokenization | 2 | HuggingFace Tokenizers, SentencePiece | | Fine-Tuning | 4 | Axolotl, LLaMA-Factory, PEFT, Unsloth | | Mech Interp | 4 | TransformerLens, SAELens, pyvene, nnsight | | Data Processing | 2 | NeMo Curator, Ray Data | | Post-Training | 8 | TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge | | Safety | 4 | Constitutional AI, LlamaGuard, NeMo Guardrails, Prompt Guard | | Distributed | 6 | DeepSpeed, FSDP, Accelerate, Megatron-Core, Lightning, Ray Train | | Infrastructure | 3 | Modal, Lambda Labs, SkyPilot | | Optimization | 6 | Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF | | Evaluation | 3 | lm-eval-harness, BigCode, NeMo Evaluator | | Inference | 4 | vLLM, TensorRT-LLM, llama.cpp, SGLang | | MLOps | 3 | W&B, MLflow, TensorBoard | | Agents | 4 | LangChain, LlamaIndex, CrewAI, AutoGPT | | RAG | 5 | Chroma, FAISS, Pinecone, Qdrant, Sentence Transformers | | Prompt Eng | 4 | DSPy, Instructor, Guidance, Outlines | | Observability | 2 | LangSmith, Phoenix | | Multimodal | 7 | CLIP, Whisper, LLaVA, BLIP-2, SAM, Stable Diffusion, AudioCraft | | Emerging | 6 | MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning | | Agent-Native Research Artifact | 3 | ARA Compiler, Research Manager, Rigor Reviewer |

View All 98 Skills in Details

### 🔬 Autoresearch (1 skill) — Central Orchestration Layer - **[Autoresearch](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/0-autoresearch-skill)** - Autonomous research orchestration using a two-loop architecture (inner optimization + outer synthesis). Manages the full lifecycle from literature survey to paper writing, routing to all domain-specific skills. Supports Claude Code /loop and OpenClaw heartbeat for continuous operation (390 lines + 3 refs) ### 🏗️ Model Architecture (5 skills) - **[LitGPT](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/litgpt)** - Lightning AI's 20+ clean LLM implementations with production training recipes (462 lines + 4 refs) - **[Mamba](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/mamba)** - State-space models with O(n) complexity, 5× faster than Transformers (253 lines + 3 refs) - **[RWKV](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/rwkv)** - RNN+Transformer hybrid, infinite context, Linux Foundation project (253 lines + 3 refs) - **[NanoGPT](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/nanogpt)** - Educational GPT in ~300 lines by Karpathy (283 lines + 3 refs) - **[TorchTitan](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/torchtitan)** - PyTorch-native distributed training for Llama 3.1 with 4D parallelism ### 🔤 Tokenization (2 skills) - **[HuggingFace Tokenizers](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/02-tokenization/huggingface-tokenizers)** - Rust-based, <20s/GB, BPE/WordPiece/Unigram algorithms (486 lines + 4 refs) - **[SentencePiece](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/02-tokenization/sentencepiece)** - Language-independent, 50k sentences/sec, used by T5/ALBERT (228 lines + 2 refs) ### 🎯 Fine-Tuning (4 skills) - **[Axolotl](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/03-fine-tuning/axolotl)** - YAML-based fine-tuning with 100+ models (156 lines + 4 refs) - **[LLaMA-Factory](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/03-fine-tuning/llama-factory)** - WebUI no-code fine-tuning (78 lines + 5 refs) - **[Unsloth](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/03-fine-tuning/unsloth)** - 2x faster QLoRA fine-tuning (75 lines + 4 refs) - **[PEFT](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/03-fine-tuning/peft)** - Parameter-efficient fine-tuning with LoRA, QLoRA, DoRA, 25+ methods (431 lines + 2 refs) ### 🔬 Mechanistic Interpretability (4 skills) - **[TransformerLens](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/04-mechanistic-interpretability/transformer-lens)** - Neel Nanda's library for mech interp with HookPoints, activation caching (346 lines + 3 refs) - **[SAELens](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/04-mechanistic-interpretability/saelens)** - Sparse Autoencoder training and analysis for feature discovery (386 lines + 3 refs) - **[pyvene](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/04-mechanistic-interpretability/pyvene)** - Stanford's causal intervention library with declarative configs (473 lines + 3 refs) - **[nnsight](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/04-mechanistic-interpretability/nnsight)** - Remote interpretability via NDIF, run experiments on 70B+ models (436 lines + 3 refs) ### 📊 Data Processing (2 skills) - **[Ray Data](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/05-data-processing/ray-data)** - Distributed ML data processing, streaming execution, GPU support (318 lines + 2 refs) - **[NeMo Curator](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/05-data-processing/nemo-curator)** - GPU-accelerated data curation, 16× faster deduplication (375 lines + 2 refs) ### 🎓 Post-Training (8 skills) - **[TRL Fine-Tuning](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/trl-fine-tuning)** - Transformer Reinforcement Learning (447 lines + 4 refs) - **[GRPO-RL-Training](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/grpo-rl-training)** (TRL) - Group Relative Policy Optimization with TRL (569 lines, **gold standard**) - **[OpenRLHF](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/openrlhf)** - Full RLHF pipeline with Ray + vLLM (241 lines + 4 refs) - **[SimPO](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/simpo)** - Simple Preference Optimization, no reference model needed (211 lines + 3 refs) - **[verl](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/verl)** - ByteDance's HybridFlow RL framework, FSDP/Megatron + vLLM/SGLang backends (389 lines + 2 refs) - **[slime](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/slime)** - THUDM's Megatron+SGLang framework powering GLM-4.x models (464 lines + 2 refs) - **[miles](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/miles)** - Enterprise fork of slime with FP8, INT4, speculative RL for MoE training (315 lines + 2 refs) - **[torchforge](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/06-post-training/torchforge)** - Meta's PyTorch-native RL with Monarch+TorchTitan+vLLM (380 lines + 2 refs) ### 🛡️ Safety & Alignment (4 skills) - **[Constitutional AI](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/07-safety-alignment/constitutional-ai)** - AI-driven self-improvement via principles (282 lines) - **[LlamaGuard](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/07-safety-alignment/llamaguard)** - Safety classifier for LLM inputs/outputs (329 lines) - **[NeMo Guardrails](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/07-safety-alignment/nemo-guardrails)** - Programmable guardrails with Colang (289 lines) - **[Prompt Guard](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/07-safety-alignment/prompt-guard)** - Meta's 86M prompt injection & jailbreak detector, 99%+ TPR, <2ms GPU (313 lines) ### ⚡ Distributed Training (6 skills) - **[Megatron-Core](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/megatron-core)** - NVIDIA's framework for training 2B-462B param models with 47% MFU on H100 (359 lines + 4 refs) - **[DeepSpeed](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/deepspeed)** - Microsoft's ZeRO optimization (137 lines + 9 refs) - **[PyTorch FSDP2](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/pytorch-fsdp2)** - Fully Sharded Data Parallel v2 with `fully_shard` and DTensor (231 lines + 12 refs) - **[Accelerate](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/accelerate)** - HuggingFace's 4-line distributed training API (324 lines + 3 refs) - **[PyTorch Lightning](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/pytorch-lightning)** - High-level training framework with Trainer class (339 lines + 3 refs) - **[Ray Train](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/08-distributed-training/ray-train)** - Multi-node orchestration and hyperparameter tuning (399 lines + 1 ref) ### 🚀 Optimization (6 skills) - **[Flash Attention](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/flash-attention)** - 2-4x faster attention with memory efficiency (359 lines + 2 refs) - **[bitsandbytes](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/bitsandbytes)** - 8-bit/4-bit quantization for 50-75% memory reduction (403 lines + 3 refs) - **[GPTQ](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/gptq)** - 4-bit post-training quantization, 4× memory reduction, <2% accuracy loss (443 lines + 3 refs) - **[AWQ](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/awq)** - Activation-aware weight quantization, 4-bit with minimal accuracy loss (310 lines + 2 refs) - **[HQQ](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/hqq)** - Half-Quadratic Quantization, no calibration data needed, multi-backend (370 lines + 2 refs) - **[GGUF](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/10-optimization/gguf)** - llama.cpp quantization format, K-quant methods, CPU/Metal inference (380 lines + 2 refs) ### 📊 Evaluation (3 skills) - **[lm-evaluation-harness](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/11-evaluation/lm-evaluation-harness)** - EleutherAI's standard for benchmarking LLMs across 60+ tasks (482 lines + 4 refs) - **[BigCode Evaluation Harness](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/11-evaluation/bigcode-evaluation-harness)** - Code model benchmarking with HumanEval, MBPP, MultiPL-E, pass@k metrics (406 lines + 3 refs) - **[NeMo Evaluator](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/11-evaluation/nemo-evaluator)** - NVIDIA's enterprise platform for 100+ benchmarks across 18+ harnesses with multi-backend execution (454 lines + 4 refs) ### ☁️ Infrastructure (3 skills) - **[Modal](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/09-infrastructure/modal)** - Serverless GPU cloud with Python-native API, T4-H200 on-demand (342 lines + 2 refs) - **[SkyPilot](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/09-infrastructure/skypilot)** - Multi-cloud orchestration across 20+ providers with spot recovery (390 lines + 2 refs) - **[Lambda Labs](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/09-infrastructure/lambda-labs)** - Reserved/on-demand GPU cloud with H100/A100, persistent filesystems (390 lines + 2 refs) ### 🔥 Inference & Serving (4 skills) - **[vLLM](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/12-inference-serving/vllm)** - High-throughput LLM serving with PagedAttention (356 lines + 4 refs, **production-ready**) - **[TensorRT-LLM](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/12-inference-serving/tensorrt-llm)** - NVIDIA's fastest inference, 24k tok/s, FP8/INT4 quantization (180 lines + 3 refs) - **[llama.cpp](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/12-inference-serving/llama-cpp)** - CPU/Apple Silicon inference, GGUF quantization (251 lines + 3 refs) - **[SGLang](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/12-inference-serving/sglang)** - Structured generation with RadixAttention, 5-10× faster for agents (435 lines + 3 refs) ### 🤖 Agents (4 skills) - **[LangChain](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/14-agents/langchain)** - Most popular agent framework, 500+ integrations, ReAct pattern (658 lines + 3 refs, **production-ready**) - **[LlamaIndex](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/14-agents/llamaindex)** - Data framework for LLM apps, 300+ connectors, RAG-focused (535 lines + 3 refs) - **[CrewAI](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/14-agents/crewai)** - Multi-agent orchestration, role-based collaboration, autonomous workflows (498 lines + 3 refs) - **[AutoGPT](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/14-agents/autogpt)** - Autonomous AI agent platform, visual workflow builder, continuous execution (400 lines + 2 refs) ### 🔍 RAG (5 skills) - **[Chroma](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/15-rag/chroma)** - Open-source embedding database, local/cloud, 24k stars (385 lines + 1 ref) - **[FAISS](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/15-rag/faiss)** - Facebook's similarity search, billion-scale, GPU acceleration (295 lines) - **[Sentence Transformers](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/15-rag/sentence-transformers)** - 5000+ embedding models, multilingual, 15k stars (370 lines) - **[Pinecone](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/15-rag/pinecone)** - Managed vector database, auto-scaling, <100ms latency (410 lines) - **[Qdrant](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/15-rag/qdrant)** - High-performance vector search, Rust-powered, hybrid search with filtering (493 lines + 2 refs) ### 🎨 Multimodal (7 skills) - **[CLIP](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/clip)** - OpenAI's vision-language model, zero-shot classification, 25k stars (320 lines) - **[Whisper](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/whisper)** - Robust speech recognition, 99 languages, 73k stars (395 lines) - **[LLaVA](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/llava)** - Vision-language assistant, image chat, GPT-4V level (360 lines) - **[Stable Diffusion](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/stable-diffusion)** - Text-to-image generation via HuggingFace Diffusers, SDXL, ControlNet (380 lines + 2 refs) - **[Segment Anything](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/segment-anything)** - Meta's SAM for zero-shot image segmentation with points/boxes (500 lines + 2 refs) - **[BLIP-2](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/blip-2)** - Vision-language pretraining with Q-Former, image captioning, VQA (500 lines + 2 refs) - **[AudioCraft](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/18-multimodal/audiocraft)** - Meta's MusicGen/AudioGen for text-to-music and text-to-sound (470 lines + 2 refs) ### 🎯 Prompt Engineering (4 skills) - **[DSPy](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/16-prompt-engineering/dspy)** - Declarative prompt programming with optimizers, Stanford NLP, 22k stars (438 lines + 3 refs) - **[Instructor](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/16-prompt-engineering/instructor)** - Structured LLM outputs with Pydantic validation, 15k stars (726 lines + 3 refs) - **[Guidance](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/16-prompt-engineering/guidance)** - Constrained generation with regex/grammars, Microsoft Research, 18k stars (485 lines + 3 refs) - **[Outlines](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/16-prompt-engineering/outlines)** - Structured text with FSM, zero-overhead, 8k stars (601 lines + 3 refs) ### 📊 MLOps (3 skills) - **[Weights & Biases](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/13-mlops/weights-and-biases)** - Experiment tracking, sweeps, artifacts, model registry (427 lines + 3 refs) - **[MLflow](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/13-mlops/mlflow)** - Model registry, tracking, deployment, autologging (514 lines + 3 refs) - **[TensorBoard](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/13-mlops/tensorboard)** - Visualization, profiling, embeddings, scalars/images (538 lines + 3 refs) ### 👁️ Observability (2 skills) - **[LangSmith](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/17-observability/langsmith)** - LLM observability, tracing, evaluation, monitoring for AI apps (422 lines + 2 refs) - **[Phoenix](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/17-observability/phoenix)** - Open-source AI observability with OpenTelemetry tracing and LLM evaluation (380 lines + 2 refs) ### 🔬 Emerging Techniques (6 skills) - **[MoE Training](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/moe-training)** - Mixture of Experts training with DeepSpeed, Mixtral 8x7B, 5× cost reduction (515 lines + 3 refs) - **[Model Merging](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/model-merging)** - Combine models with TIES, DARE, SLERP using mergekit (528 lines + 3 refs) - **[Long Context](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/long-context)** - Extend context windows with RoPE, YaRN, ALiBi, 32k-128k tokens (624 lines + 3 refs) - **[Speculative Decoding](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/speculative-decoding)** - 1.5-3.6× faster inference with Medusa, Lookahead (379 lines) - **[Knowledge Distillation](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/knowledge-distillation)** - Compress models 70B→7B with MiniLLM, temperature scaling (424 lines) - **[Model Pruning](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/model-pruning)** - 50% sparsity with Wanda, SparseGPT, <1% accuracy loss (417 lines) ### 📝 ML Paper Writing (2 skills) - **[ML Paper Writing](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/20-ml-paper-writing)** - Write publication-ready papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM with LaTeX templates, citation verification, and writing best practices (532 lines + 5 refs) - **[Academic Plotting](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/20-ml-paper-writing/academic-plotting)** - Generate publication-quality figures for ML papers: architecture diagrams via Gemini AI and data-driven charts via matplotlib/seaborn with venue-specific styling (479 lines + 3 refs) ### 💡 Ideation (2 skills) - **[Research Brainstorming](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/21-research-ideation/brainstorming-research-ideas)** - Structured ideation frameworks for discovering high-impact research directions with 10 complementary lenses (384 lines) - **[Creative Thinking](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/21-research-ideation/creative-thinking-for-research)** - Cognitive science frameworks (bisociation, structure-mapping, constraint manipulation) for genuinely novel research ideas (366 lines) ### 🧬 Agent-Native Research Artifact (3 skills) - **[ARA Compiler](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/22-agent-native-research-artifact/compiler)** - Compiles any research input (PDF papers, repos, experiment logs, raw notes) into a complete Agent-Native Research Artifact with claims, exploration graph, evidence, and code stubs (245 lines + 3 refs) - **[ARA Research Manager](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/22-agent-native-research-artifact/research-manager)** - Post-task research recorder that runs at session end to extract decisions, experiments, dead ends, and pivots from conversation history into the `ara/` directory with user-vs-AI provenance tags (324 lines + 3 refs) - **[ARA Rigor Reviewer](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/22-agent-native-research-artifact/rigor-reviewer)** - ARA Seal Level 2 semantic epistemic review scoring six dimensions of research rigor (evidence relevance, falsifiability, scope, coherence, exploration integrity, methodology) with severity-ranked findings (322 lines + 1 ref)

## Demos All 98 skills in this repo are automatically synced to [Orchestra Research](https://www.orchestra-research.com/research-skills), where you can add them to your projects with one click and use them with AI research agents. **See skills in action → [demos/](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/README.md)** We maintain a curated collection of demo repositories showing how to use skills for real AI research tasks: | Demo | Skills Used | What It Does | |------|-------------|--------------| | **[Norm Heterogeneity → LoRA Brittleness](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/autoresearch-norm-heterogeneity)** | Autoresearch, ML Paper Writing, Ideation | Agent autonomously discovered norm heterogeneity predicts fine-tuning difficulty (r=-0.99), pivoting from a null result on ETF overlaps | | **[RL Algorithm Brain Scan](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/autoresearch-rl-brain-scan)** | Autoresearch, GRPO, TRL, SAELens, TransformerLens, ML Paper Writing | Agent found DPO is a rank-1 perturbation (95.6% recovery from one SVD direction) while online RL is distributed and structure-preserving | | **[NeMo Eval: GPQA Benchmark](https://github.com/zechenzhangAGI/Nemo-Eval-Skill-Demo)** | NeMo Evaluator | Compare Llama 8B/70B/405B on graduate-level science questions | | **[LoRA Without Regret Reproduction](https://www.orchestra-research.com/perspectives/LLM-with-Orchestra)** | GRPO, TRL | Reproduce SFT + GRPO RL experiments via prompting | | **[Layer-Wise Quantization Experiment](https://github.com/AmberLJC/llama-quantization-experiment)** | llama.cpp, GGUF | Investigate optimal layer precision allocation—early layers at Q8 achieve 1.9× compression with 1.3% perplexity loss | | **[Cross-Lingual Alignment Analysis](https://github.com/AmberLJC/faiss-demo)** | FAISS | Quantify how well multilingual embeddings align semantic concepts across 8 languages using FAISS similarity search | | **[Scientific Plotting Demo](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/scientific-plotting-demo)** | Academic Plotting | Generate publication-quality figures for the Andes QoE-aware LLM serving paper — Gemini AI architecture diagrams + matplotlib data charts (CDF, multi-panel grids, bar charts) | **Featured Demos**: Two papers produced entirely by AI agents using the **autoresearch** skill. The [Norm Heterogeneity paper](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/autoresearch-norm-heterogeneity) demonstrates autonomous research pivoting — the agent refuted its own hypothesis and discovered a stronger finding. The [RL Brain Scan paper](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/demos/autoresearch-rl-brain-scan) demonstrates multi-skill orchestration — the agent trained RL models, analyzed internals with interpretability tools, and synthesized the insight that "DPO is rank-1 alignment." Both papers written end-to-end by the agent. ## Skill Structure Each skill follows a battle-tested format for maximum usefulness: ``` skill-name/ ├── SKILL.md # Quick reference (50-150 lines) │ ├── Metadata (name, description, version) │ ├── When to use this skill │ ├── Quick patterns & examples │ └── Links to references │ ├── references/ # Deep documentation (300KB+) │ ├── README.md # From GitHub/official docs │ ├── api.md # API reference │ ├── tutorials.md # Step-by-step guides │ ├── issues.md # Real GitHub issues & solutions │ ├── releases.md # Version history & breaking changes │ └── file_structure.md # Codebase navigation │ ├── scripts/ # Helper scripts (optional) └── assets/ # Templates & examples (optional) ```

Quality Standards

- 300KB+ documentation from official sources - Real GitHub issues & solutions (when available) - Code examples with language detection - Version history & breaking changes - Links to official docs

## Roadmap The library spans 98 comprehensive skills across the full AI research lifecycle. See our [detailed roadmap](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/docs/ROADMAP.md) for the complete development plan. [View Full Roadmap →](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/docs/ROADMAP.md)

View Detailed Statistics

| Metric | Current | Target | |--------|---------|--------| | **Skills** | **87** (high-quality, standardized YAML) | 80 ✅ | | **Avg Lines/Skill** | **420 lines** (focused + progressive disclosure) | 200-600 lines | | **Documentation** | **~130,000 lines** total (SKILL.md + references) | 100,000+ lines | | **Gold Standard Skills** | **65** with comprehensive references | 50+ | | **Contributors** | 1 | 100+ | | **Coverage** | Architecture, Tokenization, Fine-Tuning, Mechanistic Interpretability, Data Processing, Post-Training, Safety, Distributed, Optimization, Evaluation, Infrastructure, Inference, Agents, RAG, Multimodal, Prompt Engineering, MLOps, Observability, ML Paper Writing, Ideation, Autoresearch | Full Lifecycle ✅ | **Recent Progress**: npm package `@orchestra-research/ai-research-skills` for one-command installation across all coding agents **Philosophy**: Quality > Quantity. Following [Anthropic official best practices](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/anthropic_official_docs/best_practices.md) - each skill provides 200-500 lines of focused, actionable guidance with progressive disclosure.

## Repository Structure ``` claude-ai-research-skills/ ├── README.md ← You are here ├── CONTRIBUTING.md ← Contribution guide ├── demos/ ← Curated demo gallery (links to demo repos) ├── docs/ ├── 0-autoresearch-skill/ (1 skill ✓ - Autonomous research orchestration) ├── 01-model-architecture/ (5 skills ✓ - LitGPT, Mamba, RWKV, NanoGPT, TorchTitan) ├── 02-tokenization/ (2 skills ✓ - HuggingFace Tokenizers, SentencePiece) ├── 03-fine-tuning/ (4 skills ✓ - Axolotl, LLaMA-Factory, Unsloth, PEFT) ├── 04-mechanistic-interpretability/ (4 skills ✓ - TransformerLens, SAELens, pyvene, nnsight) ├── 05-data-processing/ (2 skills ✓ - Ray Data, NeMo Curator) ├── 06-post-training/ (8 skills ✓ - TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge) ├── 07-safety-alignment/ (4 skills ✓ - Constitutional AI, LlamaGuard, NeMo Guardrails, Prompt Guard) ├── 08-distributed-training/ (6 skills ✓ - Megatron-Core, DeepSpeed, FSDP, Accelerate, Lightning, Ray Train) ├── 09-infrastructure/ (3 skills ✓ - Modal, SkyPilot, Lambda Labs) ├── 10-optimization/ (6 skills ✓ - Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF) ├── 11-evaluation/ (3 skills ✓ - lm-evaluation-harness, BigCode, NeMo Evaluator) ├── 12-inference-serving/ (4 skills ✓ - vLLM, TensorRT-LLM, llama.cpp, SGLang) ├── 13-mlops/ (3 skills ✓ - Weights & Biases, MLflow, TensorBoard) ├── 14-agents/ (4 skills ✓ - LangChain, LlamaIndex, CrewAI, AutoGPT) ├── 15-rag/ (5 skills ✓ - Chroma, FAISS, Sentence Transformers, Pinecone, Qdrant) ├── 16-prompt-engineering/ (4 skills ✓ - DSPy, Instructor, Guidance, Outlines) ├── 17-observability/ (2 skills ✓ - LangSmith, Phoenix) ├── 18-multimodal/ (7 skills ✓ - CLIP, Whisper, LLaVA, Stable Diffusion, SAM, BLIP-2, AudioCraft) ├── 19-emerging-techniques/ (6 skills ✓ - MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning) ├── 20-ml-paper-writing/ (2 skills ✓ - ML Paper Writing with LaTeX templates, Academic Plotting) ├── 21-research-ideation/ (2 skills ✓ - Research Brainstorming, Creative Thinking) ├── 22-agent-native-research-artifact/ (3 skills ✓ - ARA Compiler, Research Manager, Rigor Reviewer) └── packages/ai-research-skills/ (npm package for one-command installation) ``` ## Use Cases ### For Researchers "I need to fine-tune Llama 3 with custom data" → **03-fine-tuning/axolotl/** - YAML configs, 100+ model support ### For ML Engineers "How do I optimize inference latency?" → **12-inference-serving/vllm/** - PagedAttention, batching ### For Students "I want to learn how transformers work" → **01-model-architecture/litgpt/** - Clean implementations ### For Teams "We need to scale training to 100 GPUs" → **08-distributed-training/deepspeed/** - ZeRO stages, 3D parallelism ## License MIT License - See [LICENSE](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/LICENSE) for details. **Note**: Individual skills may reference libraries with different licenses. Please check each project's license before use. ## Citation If you use AI Research Skills in your work or find it helpful for a publication, we'd appreciate a citation: **BibTeX** ```bibtex @software{ai_research_skills, title = {AI Research Skills Library}, author = {{Orchestra Research}}, year = {2025}, url = {https://github.com/orchestra-research/AI-research-SKILLs}, note = {Open-source skills library enabling AI agents to autonomously conduct AI research} } ``` **APA** > Orchestra Research. (2025). *AI Research Skills Library* [Computer software]. https://github.com/orchestra-research/AI-research-SKILLs **Chicago** > Orchestra Research. "AI Research Skills Library." GitHub, 2025. https://github.com/orchestra-research/AI-research-SKILLs. **IEEE** > Orchestra Research, "AI Research Skills Library," 2025. [Online]. Available: https://github.com/orchestra-research/AI-research-SKILLs > **Tip**: You can also click **"Cite this repository"** in the GitHub sidebar for auto-formatted citations. ## Acknowledgments Built with: - **[Claude Code](https://www.claude.com/product/claude-code)** - AI pair programming - **[Skill Seeker](https://github.com/yusufkaraaslan/Skill_Seekers)** - Automated doc scraping - **Open Source AI Community** - For amazing tools and docs Special thanks to: - EleutherAI, HuggingFace, NVIDIA, Lightning AI, Meta AI, Anthropic - All researchers who maintain excellent documentation ## Contributors Thanks to all the people who have contributed to the AI Research Skills Library:

We welcome contributions from the AI research community! See [CONTRIBUTING.md](https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/CONTRIBUTING.md) for detailed guidelines on: - Adding new skills - Improving existing skills - Quality standards and best practices - Submission process ## Recent Updates

June 2026 - v1.7.1 🧹 Inventory Consistency, Drift Guard & Security Hardening

- 📊 **Repo-wide inventory reconciled to 98 skills / 23 categories** — corrected stale counts that had drifted apart across files: CLAUDE.md (said 90), CONTRIBUTING.md (said 86/22), the README sync line (said 87), WELCOME.md and the npm package README (said 86/22). Also fixed wrong per-category listings (TorchTitan, SwanLab, A-Evolve, ML Training Recipes, Cosmos Policy/OpenPI/OpenVLA-OFT, the paper-writing skills) - 🛡️ **New CI drift guard** — `scripts/check-inventory.sh` + `check-inventory.yml` fail CI whenever the documented skill/category counts diverge from the actual `SKILL.md` count on disk, so the inventory can't silently drift again - 📦 **Marketplace sync hardening** — `sync-skills.yml` now prunes build artifacts (`node_modules`/`__pycache__`/`*.pyc`/`.ipynb_checkpoints`) before zipping and fails loudly above 190 files instead of hitting the marketplace's 200-file rejection > The README is long; the above is its first 40000 characters. Full content: https://github.com/Orchestra-Research/AI-Research-SKILLs