Life Sciences NGS Analysis

Life Sciences NGS Analysis Plugin

This plugin provides a guided intake and execution layer for common next-generation sequencing analyses. It routes users from BCL or FASTQ files to public, reproducible pipelines while checking local tool availability before installing anything.

What It Does

• Inspects sequencing inputs before asking questions.
• Asks the minimum assay-specific questions needed to choose an analysis route.
• Prefers public, runtime-installable tools and nf-core workflows where practical.
• Runs tool preflight checks before suggesting downloads or installs.
• Keeps proprietary, credentialed, or cloud-upload paths explicit instead of silently using them.
• Treats preflight as validation before executing approved local workflows where supported.
• Produces timestamped run directories with manifests, validation summaries, logs, QC reports, exact command timing/return-code detail, checksummed artifact indexes, and input-to-output lineage tables.
• Produces native visualization bundles under visualizations/ when a lane has enough downstream data to plot.

Included Skills

• ngs-analysis-router: top-level intake and routing.
• ngs-runtime-env: package/tool existence checks and install planning.
• ngs-bcl-to-fastq: BCL run-folder validation, demultiplexing, and demux metric review.
• ngs-fastq-qc: FASTQ quality control, trimming decisions, and MultiQC interpretation.
• ngs-dna-variant-calling: WGS/WES/panel variant dispatcher.
• ngs-dna-germline-variants: germline WGS/WES/panel variant calling and QC.
• ngs-dna-somatic-variants: tumor-normal and tumor-only somatic variant calling and QC.
• ngs-dna-umi-panel-variants: UMI, duplex, and low-frequency targeted panel workflows.
• ngs-bulk-rnaseq: bulk RNA-seq dispatcher.
• ngs-bulk-rnaseq-counts-qc: bulk RNA-seq FASTQ-to-count processing and QC.
• ngs-bulk-rnaseq-differential-expression: bulk RNA-seq count-matrix differential expression.
• ngs-scrna-seq: single-cell or single-nucleus RNA-seq FASTQ-to-count kickoff.
• scrna-seq-qc: embedded post-count single-cell QC, annotation, clustering, and UMAP guidance.
• ngs-epigenomics-peaks: ATAC-seq, ChIP-seq, CUT&RUN, and CUT&Tag dispatcher.
• ngs-atacseq-peaks-qc: ATAC-seq QC, peak, consensus, and differential accessibility workflows.
• ngs-chip-cutrun-peaks-qc: ChIP-seq, CUT&RUN, and CUT&Tag QC, control, peak, and differential binding workflows.
• ngs-amplicon-microbiome: 16S/18S/ITS/COI amplicon analysis kickoff.
• ngs-shotgun-metagenomics: shotgun metagenomics taxonomic and functional profiling kickoff.

Capability Status

This package is intentionally mixed maturity. Use the status below when deciding what to run versus what to treat as planning guidance.

Local execution lanes:

• ngs-fastq-qc: plugin-owned local runner for FASTQ validation, FastQC/MultiQC execution, optional trimming, logs, summaries, and artifact indexes.
• ngs-bulk-rnaseq-counts-qc: plugin-owned local runner for bulk RNA-seq FASTQ validation, FastQC/MultiQC, Salmon transcript quantification, TPM/NumReads/effective-length matrices, logs, summaries, and artifact indexes.
• ngs-bulk-rnaseq-differential-expression: plugin-owned local runner for count-matrix validation, contrast/replicate checks, automatic DESeq2/edgeR/limma method selection, QC plots, normalized matrices, result tables, logs, summaries, and artifact indexes.
• ngs-scrna-seq: plugin-owned local FASTQ-to-count runner for STARsolo-backed scRNA/snRNA count generation.
• scrna-seq-qc: post-count QC and annotation guidance, plus a matrix-level runner for 10x-style matrices. The runner uses conservative PBMC marker fallback when no matched reference is provided, so tissue-specific annotation should be reviewed or replaced before broader use.
• ngs-dna-variant-calling: plugin-owned BAM/CRAM-to-VCF execution package using samtools/bcftools for focused local checks, with nf-core/sarek still preferred for full WGS/WES/panel workflows.
• ngs-dna-germline-variants: plugin-owned higher-fidelity germline runner for BQSR, per-sample gVCFs, and optional joint genotyping when a local GATK toolchain and matched known-sites resources are available.
• ngs-epigenomics-peaks: plugin-owned FASTQ validation/QC execution package for ATAC-seq, ChIP-seq, CUT&RUN, and CUT&Tag intake, with readiness artifacts for the alignment and peak-calling stage.
• ngs-amplicon-microbiome: plugin-owned FASTQ validation/QC execution package for marker-gene amplicon inputs, with explicit primer/taxonomy backend readiness artifacts.
• ngs-shotgun-metagenomics: plugin-owned FASTQ validation/QC execution package for shotgun metagenomics inputs, with explicit database-gated taxonomic profiling status.
• ngs-bcl-to-fastq: plugin-owned BCL run-folder and sample-sheet validator that executes BCL Convert or legacy bcl2fastq when an installed converter is available.

Dispatch lanes:

• ngs-bulk-rnaseq: routes users to the counts/QC runner when starting from FASTQs, or to the differential-expression runner when starting from an expression matrix.

Dispatch and subtype lanes:

• ngs-dna-germline-variants
• ngs-dna-somatic-variants
• ngs-dna-umi-panel-variants
• ngs-atacseq-peaks-qc
• ngs-chip-cutrun-peaks-qc

These lanes route to the shared DNA or epigenomics execution packages when a compact local run is appropriate, and remain responsible for assay-specific guidance, metadata checks, controls, and full-workflow handoff.

Runtime Preflight

From the repo root:

python plugins/ngs-analysis/scripts/ngs_preflight.py --list
python plugins/ngs-analysis/scripts/ngs_preflight.py --pipeline bulk_rnaseq --emit-install-plan
python plugins/ngs-analysis/scripts/ngs_preflight.py --pipeline bulk_rnaseq_counts_qc --emit-install-plan
python plugins/ngs-analysis/scripts/ngs_preflight.py --pipeline bulk_rnaseq_differential_expression --emit-install-plan
python plugins/ngs-analysis/scripts/ngs_preflight.py --profile local_light --emit-install-plan
python plugins/ngs-analysis/scripts/ngs_preflight.py --tool fastqc --network-checks
python plugins/ngs-analysis/scripts/ngs_preflight.py --pipeline shotgun_metagenomics --manager micromamba --install-plan-outdir runtime_readiness/shotgun_install

The script checks local executables first with PATH lookup. Optional network checks query package indexes or container registries only when requested. It does not install packages unless --install-missing --yes is explicitly provided.

Use --install-plan-outdir for a permission-ready package handoff. It writes install_plan.json as the canonical review artifact and a guarded install_commands.sh companion generated from the same plan. The shell script exits in review-only mode unless NGS_RUN_INSTALL_COMMANDS=1 is set after explicit user approval. Reference and database downloads remain separate and should be handled through ngs_reference_manager.py setup-plan.

Pipeline Dependency Matrix

Use this matrix as a quick lane-to-package guide before running a workflow. The structured source of truth remains references/pipeline-registry.json; use ngs_preflight.py --pipeline <pipeline> --emit-install-plan to check the current machine and generate a reviewable install plan.

Pipeline lane	Required or primary tools	Useful optional tools	Reference/database requirements
BCL-to-FASTQ	bcl-convert	bcl2fastq for legacy compatibility	Illumina run folder with RunInfo.xml, optional RunParameters.xml, Data/Intensities/BaseCalls, and a compatible sample sheet
FASTQ QC and trimming	snakemake, fastqc, multiqc	fastp, cutadapt, seqkit	FASTQ files and sample sheet or explicit sample/R1/R2 paths
Bulk RNA-seq counts/QC	snakemake, fastqc, multiqc, salmon	star, subread, seqkit	Transcriptome FASTA, genome FASTA, GTF, and optional registered genome bundle
Bulk RNA-seq differential expression	Rscript plus available DE backend packages	DESeq2, edgeR, limma, marimo review surface	Count/expression matrix, sample metadata, contrast table, and design context
scRNA FASTQ-to-count	snakemake, STAR or configured STAR container	kb-python, cellranger when explicitly requested and licensed	Genome FASTA, GTF, chemistry/whitelist inputs, and optional registered genome bundle
scRNA post-count QC	Python/R analysis environment with scanpy and runner dependencies	Bioconductor QC packages, marimo review surface	10x-style matrix bundle, barcodes/features, metadata, and optional raw droplet matrix
Generic DNA variant calling	samtools, bcftools	bwa-mem2, gatk, deepvariant container	Reference FASTA, indexes, optional target/region, optional annotation VCF
Germline DNA variants	gatk, samtools	bcftools, bwa-mem2, deepvariant container	Matched reference FASTA, known-sites resources, optional target BED, optional cohort/joint-calling resources
Somatic DNA variants	gatk, samtools, bcftools	panel-of-normals resources	Tumor-normal or tumor-only sample sheet, reference FASTA, germline resource, optional panel of normals and target BED
UMI panel variants	samtools, bcftools, fgbio when consensus generation is needed	bwa-mem2, gatk, duplex-review helpers	Reference FASTA, target BED, UMI read structure/tags, consensus or raw UMI inputs, optional hotspot/review VCF
ATAC-seq peaks/QC	bowtie2, samtools, bedtools, macs2, bamCoverage	computeMatrix, plotProfile, plotHeatmap, homer, multiqc	Bowtie2 index, genome size, blacklist BED, optional TSS BED, optional registered genome bundle
ChIP/CUT&RUN/CUT&Tag peaks/QC	bowtie2, samtools, bedtools, macs2, bamCoverage	homer, multiqc	Bowtie2 index, genome size, IP/control metadata, optional blacklist/TSS annotations and registered genome bundle
Amplicon microbiome	qiime2 or dada2	cutadapt, seqkit, multiqc	Primer/marker metadata, taxonomy classifier or database, sample metadata, optional ASV/taxonomy tables for review
Shotgun metagenomics	kraken2, bracken by default	kneaddata, humann, metaphlan, seqkit, multiqc	Kraken2 database, optional host-depletion reference, optional Bracken and HUMAnN databases
nf-core adapter	nextflow	multiqc, Docker/Singularity/Apptainer or site profile	Pipeline-specific reference/database bundle and any required container/runtime profile

Package checks and reference/database checks are intentionally separate. Missing executables should produce an install plan; missing references or databases should produce resource readiness and setup-plan artifacts before any large download is attempted.

FASTQ QC Local Execution

The FASTQ QC lane accepts a sample sheet or a single sample, validates FASTQ structure and pairing, runs FastQC/MultiQC through a local Snakemake workflow, and writes a standardized run envelope. By default, outputs are written under ngs_runs/fastq_qc/ in the current working directory; pass --outdir to choose a different run directory.

python plugins/ngs-analysis/scripts/run_fastq_qc.py \
  --sample-sheet samplesheet.csv \
  --execute

Single paired sample:

python plugins/ngs-analysis/scripts/run_fastq_qc.py \
  --sample sampleA \
  --r1 sampleA_R1.fastq.gz \
  --r2 sampleA_R2.fastq.gz \
  --execute

Optional trimming and re-QC:

python plugins/ngs-analysis/scripts/run_fastq_qc.py \
  --sample-sheet samplesheet.csv \
  --trim-mode fastp \
  --execute

Each successful run writes run_manifest.json, config.json, validation/, workflow/Snakefile, logs/, artifact_index.json, summary.md, FastQC/MultiQC outputs, browser helpers plus visualizations/localhost_launch_hint.txt for the preferred localhost review path, and qc_interpretation.json.

Bulk RNA-seq Counts/QC Local Execution

The bulk RNA-seq counts/QC lane accepts an nf-core-style sample sheet, validates FASTQ paths and read structure, runs FastQC/MultiQC plus Salmon transcript quantification through a plugin-owned Snakemake workflow, and writes the standard run envelope.

python plugins/ngs-analysis/scripts/run_bulk_rnaseq_counts_qc.py \
  --sample-sheet samplesheet.csv \
  --fastq-root path/to/fastqs \
  --transcriptome-fasta reference/transcriptome.fasta \
  --genome-fasta reference/genome.fa \
  --annotation-gtf reference/genes.gtf \
  --execute

Each successful run writes run_manifest.json, config.json, validation/, workflow/Snakefile, logs/, versions/software_versions.json, artifact_index.json with per-file SHA256 checksums, summary.md, a review bundle under visualizations/, browser helpers plus visualizations/localhost_launch_hint.txt for the preferred localhost review path, Salmon quant.sf files, and rnaseq_salmon/matrices/{tpm,num_reads,effective_length,samples}.tsv.

The counts/QC runner also writes a run-local resource-readiness bundle under resources/. It is advisory by default for explicitly supplied local references; use --genome-build, --bundle-root <bundle>=<path>, and --require-resource-plan when a registered genome bundle must be complete before the run can be marked ready.

Bulk RNA-seq Differential Expression Local Execution

The differential-expression lane accepts a count or expression matrix, sample metadata, and a contrast table. It validates sample matching, replicate sufficiency, matrix scale, and R/Bioconductor package availability. With --method auto, integer-like raw_counts prefer DESeq2 when available, then edgeR; non-integer inputs route to limma_log2. Use --input-mode to declare raw_counts, normalized_expression, or log_expression; auto infers the mode and emits a warning when normalization is skipped because the matrix is already transformed.

python plugins/ngs-analysis/scripts/run_bulk_rnaseq_de.py \
  --count-matrix count_matrix.tsv \
  --sample-metadata sample_metadata.tsv \
  --contrasts contrasts.tsv \
  --input-mode auto \
  --execute

Each successful run writes run_manifest.json, config.json, validation/, workflow/scripts/run_bulk_de.R, logs/, manifest/contrast_status.tsv, input-mode-aware matrix artifacts, qc/design_matrix.tsv, qc/design_diagnostics.tsv, qc/sample_outlier_metrics.tsv, qc/statistical_warnings.tsv, qc/mean_variance_trend.png, per-contrast result tables, explicit .not_tested.tsv stubs for blocked contrasts, clearer limma volcano/MA plots when applicable, a review bundle under visualizations/, notebooks/bulk_rnaseq_de_review.marimo.py, an auto-launched localhost Marimo review app recorded in notebooks/marimo_server.json, versions/, checksummed artifact_index.json, and summary.md.

scRNA FASTQ-to-count Local Execution

The plugin-owned scRNA execution lane accepts local FASTQs, validates barcode-versus-cDNA pairing, runs STARsolo through a dedicated Snakemake workflow, and writes a standardized run envelope. By default, outputs are written under ngs_runs/scrnaseq_fastq_to_count/ in the current working directory; pass --outdir to choose a different run directory.

python plugins/ngs-analysis/scripts/run_scrnaseq_fastq_to_count.py \
  --sample-sheet samplesheet.csv \
  --genome-fasta reference/genome.fa \
  --annotation-gtf reference/genes.gtf \
  --cb-whitelist reference/whitelist.txt \
  --execute

Each successful run writes run_manifest.json, manifest/lineage.tsv, manifest/working_samplesheet.csv, manifest/inputs_manifest.tsv, config.json, validation/, workflow/Snakefile, logs/, versions/software_versions.json, artifact_index.json, summary.md, and STARsolo count artifacts. The run manifest records pinned STAR image metadata, chemistry-detection evidence, and explicit STARsolo cell-calling filter settings.

The FASTQ-to-count runner also writes advisory resources/resource_plan.json, resource_manifest.tsv, resource_env.sh, resource_readiness.md, and resource setup-plan artifacts by default. Add --require-resource-plan with a registered --genome-build and --bundle-root when genome bundle completeness should block readiness.

scRNA Post-count Execution

For 10x-style matrix bundles, the package includes a post-count QC runner:

python plugins/ngs-analysis/scripts/run_scrnaseq_post_count_qc.py \
  --input-dir path/to/scrna_bundle

The input directory should contain matrix/, manifest.tsv, and dataset_metadata.json, unless explicit paths are provided. An optional --raw-matrix-dir enables emptyDrops-style cell-calling checks when a raw droplet matrix is available. This runner emits a standard envelope with run_manifest.json, manifest/lineage.tsv, validation/tool_preflight.json, versions/software_versions.json, checksummed artifact_index.json, summary.md, provenance/analysis_status.json, and visualizations/index.html. It also auto-launches a localhost Marimo review app recorded in notebooks/marimo_server.json and emits notebooks/scrna_qc_review.marimo.py as a notebook backup over the portable PNG/CSV/H5AD outputs. Tissue-specific annotation and integration choices should still be reviewed against the dataset and reference context.

DNA Variant Calling Execution

The DNA variant-calling execution package accepts a BAM/CRAM sample sheet plus a matching reference FASTA, validates alignment/reference inputs, runs samtools QC, calls variants with bcftools, and writes the standard run envelope. Use nf-core/sarek or a lab-validated workflow for full germline, somatic, trio, or panel analysis.

python plugins/ngs-analysis/scripts/run_dna_variant_calling.py \
  --sample-sheet dna_samples.tsv \
  --reference-fasta reference.fa \
  --region chr20:1-100000 \
  --filter-min-qual 30 \
  --filter-min-site-dp 10 \
  --execute

Add a small known-variant annotation layer by passing a bgzip/tabix-indexed resource VCF:

python plugins/ngs-analysis/scripts/run_dna_variant_calling.py \
  --sample-sheet dna_samples.tsv \
  --reference-fasta reference.fa \
  --annotation-vcf gnomad_small.vcf.gz \
  --execute

Each successful run writes run_manifest.json, validation/, logs/, qc/*.flagstat.txt, qc/*.idxstats.tsv, qc/*.coverage.tsv, qc/*.depth.tsv when a region is provided, qc/*.callability.json, qc/*.variant_summary.json, variants/*.vcf.gz, optional variants/*.annotated.vcf.gz, optional variants/*.filtered.vcf.gz, variants/*.bcftools_stats.txt, artifact_index.json, and summary.md.

The generic BAM/CRAM-to-VCF runner now emits advisory resources/ readiness artifacts for the selected genome bundle. Use --require-resource-plan when missing registered references should block readiness; otherwise the explicit --reference-fasta remains enough for focused local checks.

Germline DNA Variant Calling Execution

For germline-specific local runs that should own BQSR and cohort assumptions, use the dedicated runner:

python plugins/ngs-analysis/scripts/run_dna_germline_variants.py \
  --sample-sheet dna_samples.tsv \
  --reference-fasta reference.fa \
  --known-sites dbsnp.vcf.gz \
  --known-sites mills.vcf.gz \
  --emit-gvcf \
  --joint-call \
  --execute

The runner validates resource completeness and writes a standard run envelope even when execution is blocked by missing GATK or mismatched resource bundles. Successful runs emit per-sample recalibration tables and BAMs, gvcf/*.g.vcf.gz, optional joint/cohort.joint.vcf.gz, qc/*.flagstat.txt, qc/*.idxstats.tsv, artifact_index.json, and summary.md.

The germline runner also writes the same advisory resources/ bundle by default, with --require-resource-plan available for runs that must prove a complete registered reference and known-sites bundle.

Somatic And UMI DNA Variant Execution

Somatic and UMI panel lanes now have dedicated runners instead of relying on the generic BAM-to-VCF path.

python plugins/ngs-analysis/scripts/run_dna_somatic_variants.py \
  --sample-sheet somatic_pairs.tsv \
  --reference-fasta reference.fa \
  --germline-resource af-only-gnomad.vcf.gz \
  --panel-of-normals pon.vcf.gz \
  --execute

The somatic runner validates tumor-normal/tumor-only pairing, writes workflow/somatic_command_plan.json, emits Mutect2/contamination/filtering command plans, records tumor-only and missing-resource caveats, and produces filtered VCF artifacts when GATK resources are available. It also writes qc/somatic_pair_review.{tsv,json} so each pair has an explicit matched-normal status, PON/germline-resource status, contamination-table status, filtered-VCF status, and parsed variant-count summary when backend artifacts exist.

python plugins/ngs-analysis/scripts/run_dna_umi_panel_variants.py \
  --sample-sheet umi_panel_samples.tsv \
  --reference-fasta reference.fa \
  --target-bed panel_targets.bed \
  --umi-mode duplex \
  --umi-tag RX \
  --execute

The UMI runner validates raw versus consensus BAM state, writes workflow/umi_panel_command_plan.json, emits fgbio consensus and consensus-BAM variant-calling commands, and records qc/umi_postrun_summary.{tsv,json} with consensus read counts, target coverage, variant stats, and family-size/duplex metrics when those backend artifacts exist. It also writes qc/umi_molecular_evidence_contract.{tsv,json} to make the low-AF review contract explicit: consensus BAM, family-size/molecule metrics, consensus VCF, variant stats, hotspot review, and duplex review readiness stay visible per sample.

Somatic and UMI direct runners now write advisory resource-readiness bundles by default under resources/. Use --genome-build, --bundle-root grch38_core=/refs/GRCh38, and --require-resource-plan when the run should be blocked unless the registered reference bundle is complete; leave the default advisory mode for custom or reduced references where the explicit FASTA/BED inputs are enough for a local check.

FASTQ Assay Execution

The epigenomics, amplicon microbiome, and shotgun metagenomics execution packages share a FASTQ validation/QC runner. It resolves sample-sheet paths, validates read structure, runs seqkit stats and FastQC/MultiQC when available, then writes lane-specific readiness/status artifacts for the next workflow stage.

python plugins/ngs-analysis/scripts/run_fastq_assay_package.py \
  --lane epigenomics_peaks \
  --sample-sheet assay_samples.csv \
  --execute

Supported lanes are epigenomics_peaks, amplicon_microbiome, and shotgun_metagenomics. The shotgun lane can run Kraken2 only when both kraken2 and a database path are provided; otherwise it records the database/tool blocker explicitly.

Each run also writes visualizations/index.html and visualizations/visualization_manifest.json. Successful FASTQ-assay executions emit qc_verdict.json; shotgun and amplicon runs also emit qc_interpretation.json with machine-readable reason codes, readiness verdicts, and concrete follow-on commands for backend generation plus plot re-rendering. The common run_manifest.json includes audit metadata such as plugin version, exact argv, environment snapshot, input-file checksums, a parameter hash, and manifest/lineage.tsv. With FASTQ-only inputs, the visual bundle points to read-level QC and the lane readiness artifact. When downstream tables are available, the runner can also create native plot/table bundles:

python plugins/ngs-analysis/scripts/run_fastq_assay_package.py \
  --lane amplicon_microbiome \
  --sample-sheet amplicon_samples.tsv \
  --asv-table asv_table.tsv \
  --taxonomy-table taxonomy.tsv \
  --execute

Amplicon visualizations include alpha diversity tables/plots, Bray-Curtis PCoA, rarefaction curves, and taxa barplots. Shotgun visualizations can be generated from Kraken reports, Bracken tables, and HUMAnN path/gene-family tables:

python plugins/ngs-analysis/scripts/run_fastq_assay_package.py \
  --lane shotgun_metagenomics \
  --sample-sheet shotgun_samples.csv \
  --kraken-report sample.report.txt \
  --bracken-table sample.bracken.tsv \
  --humann-pathabundance humann_pathabundance.tsv \
  --humann-genefamilies humann_genefamilies.tsv \
  --execute

For amplicon lanes, the runner also emits methods/amplicon_methods.json and a concrete backend handoff bundle under workflow/amplicon_backend_*.{json,sh}. If downstream ASV/taxonomy inputs are labeled synthetic or introduce sample columns that are not present in the real sample sheet, the run is marked review-only and beta-diversity/PCoA are blocked unless --allow-synthetic-diversity is passed explicitly.

Assay Backend Execution

Dedicated backend runners expand beyond read-QC/readiness packages when the required tools and references are present:

python plugins/ngs-analysis/scripts/run_atacseq_peaks_qc.py \
  --sample-sheet atac_samples.csv \
  --bowtie2-index /refs/GRCh38/bowtie2/genome \
  --genome-size hs \
  --blacklist-bed /refs/GRCh38/blacklists/encode_blacklist.bed \
  --tss-bed /refs/GRCh38/tss.bed \
  --execute

python plugins/ngs-analysis/scripts/run_chip_cutrun_peaks_qc.py \
  --sample-sheet chip_samples.csv \
  --assay chipseq \
  --target-class tf \
  --peak-mode narrow \
  --bowtie2-index /refs/GRCh38/bowtie2/genome \
  --genome-size hs \
  --execute

These runners produce command plans and envelopes for alignment, filtering, MACS2 peaks, FRiP, consensus peaks, bigWig tracks, ATAC TSS matrices, and motif handoff artifacts.

Epigenomics backend runs also write normalized review outputs: qc/atacseq_qc_summary.{tsv,json} or qc/chip_cutrun_qc_summary.{tsv,json}, native dashboards under qc/*_dashboard.html, compact SVG plots for FRiP/peak counts and insert-size distributions, tracks/browser_tracks.tsv, tracks/browser_track_preview.html, tracks/ucsc_track_lines.txt, tracks/igv_session.xml, and motifs/motif_summary.tsv. ATAC runs generate TSS profile/heatmap commands when --tss-bed is supplied; ATAC and ChIP/CUT&RUN runs can add HOMER motif commands with --run-motifs --motif-genome <hg38|mm10|...>.

ATAC and ChIP/CUT&RUN direct runners also write advisory resource-readiness bundles by default. Use --require-resource-plan with an explicit --genome-build and --bundle-root when reference completeness should block execution readiness.

python plugins/ngs-analysis/scripts/run_amplicon_microbiome.py \
  --sample-sheet amplicon_samples.tsv \
  --backend qiime2 \
  --primer-forward GTGYCAGCMGCCGCGGTAA \
  --primer-reverse GGACTACNVGGGTWTCTAAT \
  --taxonomy-classifier silva-138-classifier.qza \
  --metadata sample_metadata.tsv \
  --execute

python plugins/ngs-analysis/scripts/run_shotgun_metagenomics.py \
  --sample-sheet shotgun_samples.csv \
  --kraken-db /db/kraken2/standard \
  --host-reference /refs/human_kneaddata_db \
  --run-bracken \
  --run-humann \
  --humann-db /db/humann \
  --execute

Use --backend dada2 when the user wants the direct R/Bioconductor path. The plugin now ships workflows/amplicon_microbiome/run_dada2_backend.R, checks the dada2 R package before execution, runs ASV inference when the package is available, and writes tables/asv_table.tsv, tables/representative_sequences.fasta, tables/read_retention.tsv, optional tables/taxonomy.tsv, and dada2/dada2_backend_state.rds.

The amplicon runner keeps database/tool blockers explicit, exports QIIME2 denoising stats, normalizes QIIME2 exports, and preserves native DADA2 outputs in the same review contract. BIOM-only exports are reported with a concrete biom convert command rather than silently treated as parsed tables. When a normalized ASV/feature table is present, the runner now derives tables/alpha_diversity.tsv, tables/bray_curtis_distance.tsv, tables/top_taxa_or_features.tsv, tables/amplicon_diversity_summary.json, and native SVG/HTML review artifacts under visualizations/.

The amplicon direct runner writes advisory taxonomy-database readiness bundles by default. Use --bundle-root silva_138_amplicon=/db/silva, --include-optional-resources, and --require-resource-plan when database completeness should block readiness.

The shotgun runner keeps database/tool blockers explicit, runs KneadData host depletion when --host-reference is supplied, routes Kraken2 and HUMAnN over the cleaned reads, and normalizes backend outputs into tables/bracken_est_reads_matrix.tsv, tables/bracken_relative_abundance_matrix.tsv, tables/humann_pathabundance_matrix.tsv, tables/humann_genefamilies_matrix.tsv, plus Bracken/HUMAnN summary JSON. It also derives tables/top_bracken_taxa.tsv, tables/top_humann_pathways.tsv, tables/top_humann_gene_families.tsv, tables/metagenomics_backend_review.json, and native dashboard/SVG review artifacts when those matrices are available. Missing database outputs remain not_available in the visualization manifest.

For direct Kraken2/Bracken/HUMAnN runs, the shotgun runner also writes resources/resource_plan.json, resources/resource_manifest.tsv, resources/resource_env.sh, resources/resource_readiness.md, and resource setup-plan artifacts. The Kraken2 database contract is always required; Bracken and HUMAnN are promoted to blocking resource checks when --run-bracken or --run-humann is requested.

nf-core Adapter

When the user wants nf-core execution, use the adapter to generate pinned Nextflow commands and capture trace/report artifacts in the standard envelope:

python plugins/ngs-analysis/scripts/run_nfcore_pipeline.py \
  --pipeline rnaseq \
  --sample-sheet samplesheet.csv \
  --profile docker \
  --revision 3.18.0 \
  --genome GRCh38 \
  --bundle-root grch38_core=/refs/GRCh38 \
  --execute

Supported adapters include rnaseq, scrnaseq, sarek, atacseq, chipseq, cutandrun, ampliseq, and taxprofiler. Each adapter now writes a run-local resource gate under resources/ before execution: resource_plan.json, resource_manifest.tsv, resource_env.sh, resource_readiness.md, resource_setup_plan.json, resource_setup_plan.tsv, resource_setup_plan.md, and resource_setup_commands.sh. Missing required reference or database bundles block the run envelope from being marked ready; use --bundle-root bundle=/path, --genome-build, and --include-optional-resources to make the readiness state explicit. Use --skip-resource-plan only for command-shape review, not for execution-readiness claims.

References And Databases

Reference/database readiness is tracked separately from executable preflight:

python plugins/ngs-analysis/scripts/ngs_reference_manager.py list
python plugins/ngs-analysis/scripts/ngs_reference_manager.py check --kind reference --bundle grch38_core --root /refs/GRCh38
python plugins/ngs-analysis/scripts/ngs_reference_manager.py explain-missing --kind database --bundle kraken2_standard --root /db/kraken2/standard
python plugins/ngs-analysis/scripts/ngs_reference_manager.py plan --pipeline shotgun_metagenomics --include-optional --outdir resource_readiness/shotgun
python plugins/ngs-analysis/scripts/ngs_reference_manager.py setup-plan --pipeline shotgun_metagenomics --include-optional --outdir resource_readiness/shotgun_setup
python plugins/ngs-analysis/scripts/ngs_reference_manager.py plan --pipeline atacseq --genome-build GRCh38 --bundle-root grch38_core=/refs/GRCh38 --outdir resource_readiness/atac
python plugins/ngs-analysis/scripts/ngs_reference_manager.py inventory --outdir resource_readiness/inventory
python plugins/ngs-analysis/scripts/ngs_reference_manager.py lock --outdir resource_readiness/lock --include-checksums
python plugins/ngs-analysis/scripts/ngs_reference_manager.py verify-lock --lockfile resource_readiness/lock/resource_lock.json --outdir resource_readiness/lock_verify --fail-on-mismatch
python plugins/ngs-analysis/scripts/ngs_reference_manager.py check-all --kind database --output resource_readiness/database_audit.json

The plan command resolves pipeline-specific required and optional bundles, checks configured roots or --bundle-root overrides, and writes resource_plan.json, resource_manifest.tsv, resource_env.sh, resource_readiness.md, plus setup-plan artifacts. This gives each analysis a concrete reference/database gate before execution. The companion setup-plan command writes only the operator setup bundle: resource_setup_plan.json, resource_setup_plan.tsv, resource_setup_plan.md, and resource_setup_commands.sh. The shell skeleton keeps setup hints commented by default so large database/reference downloads stay deliberate. The registries live in references/reference-registry.json and references/database-registry.json; they encode expected files, root environment variables, source notes, setup hints, size caveats, and license/version caveats.

The inventory command is the project-level resource surface. It checks every known reference/database bundle, accepts repeated --bundle-root bundle=/path overrides, and writes resource_inventory.json, resource_inventory.tsv, resource_env.sh, and resource_dashboard.md. Use it before broad multi-lane runs to see missing bundle files, env vars, setup hints, license notes, and which pipelines each bundle gates.

The lock command snapshots the current inventory into resource_lock.json, resource_lock.tsv, and resource_lock.md; add --include-checksums for SHA256 hashes on files below the checksum threshold. Use verify-lock before reruns or handoffs to prove that locked resources are still present and unchanged, or to produce a concrete drift report.

BCL To FASTQ Execution

The BCL package validates RunInfo.xml, optional RunParameters.xml, Data/Intensities/BaseCalls, and the sample sheet. With --execute, it runs bcl-convert when installed, falls back to bcl2fastq when available, and otherwise records the converter blocker without auto-downloading proprietary software.

Successful executions now emit a demultiplexing QC summary under qc/demux_qc_summary.json, include generated FASTQs and BCL reports in artifact_index.json, and surface Docker-wrapper readiness in validation/runtime_preflight.json when the local converter is a Docker-backed wrapper.

python plugins/ngs-analysis/scripts/run_bcl_to_fastq.py \
  --run-folder /path/to/run \
  --sample-sheet SampleSheet.csv \
  --output-directory fastq_out \
  --execute

Local Plugin Install

The shareable unit is a marketplace root containing both:

.agents/plugins/marketplace.json
plugins/ngs-analysis/

Unzip or clone that root anywhere on the recipient machine, then add the root directory as a Codex local marketplace. Install Life Sciences NGS Analysis from the Life Sciences NGS Analysis Local Plugins marketplace. After installation, invoke it by asking for NGS routing or by referencing skills such as ngs-analysis-router, ngs-fastq-qc, ngs-dna-somatic-variants, or ngs-scrna-seq.

Do not distribute only the plugins/ngs-analysis/ directory unless the recipient already knows how to register a Codex marketplace entry for it.

Local Execution Profile

When Docker, registry egress, or Nextflow process containers are unavailable, route compact local execution runs through the local_light profile. This profile uses Snakemake or direct shell commands with isolated conda/mamba environments and avoids containers by default.

The default local lanes are FASTQ QC with FastQC/MultiQC, Salmon quantification for bulk RNA-seq inputs, bulk RNA-seq count-matrix differential expression, samtools/bcftools variant checks where suitable references exist, post-count single-cell QC, FASTQ-level epigenomics/amplicon/shotgun packages, and BCL run-folder validation/conversion when an Illumina converter is installed.

Public-First Boundary

The default registry favors public packages and workflows such as nf-core, FastQC, MultiQC, Cutadapt, fastp, STAR, Salmon, GATK4, DeepVariant, samtools/bcftools, QIIME2, DADA2, Kraken2, and HUMAnN.

Some common tools are public to download but not fully open-source or may require EULA/license acceptance. Examples include Illumina BCL Convert, 10x Cell Ranger, DRAGEN, and Sentieon. The skills surface those boundaries before use.