Tools
Alle Skills, MCP-Server, CLIs und Plugins des Verzeichnisses. Nach Typ, Beruf oder Tag filtern oder per Stichwort suchen.
Beruf: Data engineering
Einen oder mehrere wählen: Angezeigt werden Ressourcen, die zu einem davon passen.
Tags: data-pipelines
Einen oder mehrere wählen: Angezeigt werden nur Ressourcen, die alle tragen.
- data-pipelines21
- sql7
- database6
- dbt6
- data-analysis4
- python4
- csv3
- file-conversion3
- json3
- spark3
- web-scraping3
- airflow2
- databricks2
- etl2
- java2
- terminal2
- unit-testing2
- alibaba1
- aliyun1
- analytics-engineering1
- apache1
Es gibt noch mehr. Zum Suchen Text eingeben.
21 Einträge

Crawl4AI
CLIOpen-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.
unclecode/crawl4ai87.5k
Data Engineering
PluginData engineering plugin from the wshobson/agents marketplace: a data engineer subagent, a data pipeline command, and skills for Airflow DAGs, dbt transformations, Spark tuning and data quality, to help build ETL and data warehouses.
wshobson/agents42.7k
jq
CLILightweight command-line JSON processor that slices, filters, maps and transforms JSON with concise expressions and no runtime dependencies. A staple for agents parsing API responses and logs.
jqlang/jq36.2k
DataX
CLIAlibaba's open-source offline data sync tool: JSON job configs batch-sync between heterogeneous sources like MySQL, Oracle, PostgreSQL, Hive, HDFS and MaxCompute, run with python datax.py. Needs JDK 1.8+ and Python; download and unpack the official toolkit.
alibaba/DataX17.4k
Apify MCP Server
MCPLets agents call thousands of ready-made scrapers and automation Actors from Apify Store to pull data from social media, search engines, maps, e-commerce sites and more. Maintained by Apify; needs an Apify account (OAuth) or API token.
apify/apify-mcp-server16.9k
DVC
CLIVersion datasets and models the way Git versions code: data lives in remote storage like S3 or GCS while versions are tracked in your Git repo. Also offers reproducible pipelines and local experiment tracking, rerunning only the steps affected by a change.
treeverse/dvc16k
dbt
CLICommand-line framework for warehouse modeling and ELT transformations: write models in SQL and let dbt handle dependencies, data tests and documentation. v2 is rewritten in Rust as a single binary. Maintained by dbt Labs; needs a data warehouse connection.
dbt-labs/dbt14.2k
Miller
CLIProcesses CSV, TSV, JSON and other structured data like awk, sed, cut, join and sort, but by field name, for cleaning, aggregation, format conversion and pretty-printing. The command is mlr; written in Go with no runtime dependencies.
johnkerl/miller10.1k
Apache SeaTunnel
CLIDistributed data integration tool and Apache top-level project with 160+ connectors for batch, real-time and CDC sync. Define jobs in config files and run them on the built-in Zeta engine or on Flink or Spark. Requires Java.
apache/seatunnel9.8k
dasel
CLIQuery, modify and convert JSON, YAML, TOML, XML, CSV, HCL, INI and more with one selector syntax, editing config files in place or converting between formats, handy for agents handling structured data in scripts. A single dependency-free binary that works much like jq or yq.
TomWright/dasel8.1k
Trafilatura
CLIPython library and CLI for extracting main web content: fetches pages, crawls via sitemaps and RSS feeds, strips headers, footers and other noise, and outputs TXT, Markdown, JSON, XML and more with metadata. Also processes downloaded HTML offline.
adbar/trafilatura7.1k
qsv
CLIFast Rust toolkit for tabular data with nearly 80 subcommands for querying, stats, cleaning, validation, joins and format conversion on CSV and Excel. Also offers an MCP server and a Claude Cowork plugin.
dathere/qsv3.8k
SQLMesh
CLIFramework for writing data transformation models in SQL or Python, with a Terraform-like plan/apply flow, virtual data environments, unit tests and table diffs, and transpilation across 10+ SQL dialects. A Linux Foundation project and an alternative to dbt.
SQLMesh/sqlmesh3.4k
dbt Agent Skills
Skilldbt Labs' official skills: write and edit dbt models, add unit tests and docs, set up the MetricFlow semantic layer, debug failed jobs, and handle dbt version upgrades and cross-platform migrations. Most skills need dbt installed and an existing project.
dbt-labs/dbt-agent-skills765
dbt Agent Skills
Plugindbt Labs' official plugin marketplace with dbt and dbt-migration plugins: write models and unit tests, build semantic-layer metrics, govern with dbt Mesh, debug failed jobs, and handle version upgrades and cross-platform migrations. Most skills need an existing dbt project.
dbt-labs/dbt-agent-skills765
dbt MCP Server
MCPLets agents read dbt project models, lineage and Semantic Layer metrics, run build, run and test commands, and call the dbt platform Discovery and Admin APIs. Maintained by dbt Labs; Semantic Layer and platform tools need a dbt platform account.
dbt-labs/dbt-mcp624
Astronomer Data Agents
PluginAstronomer's official data engineering plugin: write Airflow DAGs to best practices, debug pipeline failures, trace data lineage, profile warehouse tables and migrate from Airflow 2 to 3, with a built-in Airflow MCP server and session hooks.
astronomer/agents463
Databricks Agent Skills
SkillOfficial Databricks skills guiding agents to build Lakeflow pipelines and jobs, Apps, Lakebase, Vector Search, Model Serving and AI/BI dashboards on the lakehouse. Needs the Databricks CLI.
databricks/databricks-agent-skills407
Databricks Agent Skills
PluginOfficial Databricks plugin with skills for the CLI and auth, Genie data queries, Apps, Lakebase, Lakeflow jobs and pipelines, Model Serving and DABs deployment, plus hooks for prompt routing, session context and auth failure hints. Needs a Databricks account.
databricks/databricks-agent-skills404
Alibaba Cloud DataWorks MCP Server
MCPAlibaba Cloud's official MCP server: through the DataWorks Open API, agents manage data development nodes and workflows, operate scheduled instances, query Data Map metadata and lineage, and configure data quality rules. Needs an Alibaba Cloud AccessKey.
aliyun/alibabacloud-dataworks-mcp-server58
Snowflake Cortex Code
PluginSnowflake's official plugin: detects Snowflake-related requests and hands them to the Cortex Code CLI, whose 55+ built-in skills handle SQL, dynamic tables, data governance and more. Needs the cortex CLI installed and a Snowflake connection configured.
Snowflake-Labs/snowflake-ai-kit45