We recently announced LakeForge - our Databricks LakeFlow pipeline framework. Part of building LakeForge involved building agents that work efficiently and actually scale well, so you can vibe-code to your heart's content knowing that there are guardrails of traditional engineering built in. If you missed it, you can read more about LakeForge here.
So we developed a set of agents designed to work together to design, validate and build LakeForge-standard pipelines. Obviously we had to give it a fun name, so Pantheon is our team of agents, each named after a Greek god. We're nerdy in more ways than one! Essentially, Pantheon exists because we wanted to remove the boring bits, standardise the clever bits, and let humans focus on the genuinely interesting bits.
Pantheon: What, Why & How It Fits With LakeForge
We’ve always loved the idea of giving data engineers superpowers… but without creating yet another mysterious black box. Instead, each AI agent in Pantheon has a specific purpose, rooted in Greek mythology. Think Athena sketching out elegant models, Hephaestus hammering pipelines into shape, Hermes weaving in custom SQL with flair, Artemis catching anything that looks suspicious, and Zeus keeping the whole celestial production line in order.
Pantheon (and LakeForge as a whole) exists because engineering teams are vibe-coding more and more - but with no standardisation and the same usual pain points (delivery bottlenecks, pattern drift, "who wrote this?" moments), we were seeing even more chaos than pre-vibe-coding days. We want to give engineers the capability of vibe-coding and the speed that comes with it, but without compromising on quality.
So how do LakeForge and Pantheon fit together?
The relationship is simple:
LakeForge is the pipeline framework. Pantheon is the workshop that creates new LakeForge pipelines.
They’re separate on purpose, but designed to operate together.
You can imagine LakeForge as our sleek new car, getting you from A to B in record time. Pantheon is our autonomous factory, churning out new cars every second.
How Pantheon Works
Pantheon works like a miniature Olympus built for data pipelines. Each agent owns specific tasks in the chain, and is equipped with a range of skills they can apply when required. The whole thing flows through a clean arc:
- Understand the request
- Profile the data
- Design the layers
- Shape and validate the logic
- Build pipeline + docs, the LakeForge way
It’s collaborative, structured, and - crucially - fun to watch in action. The end result is a consistent, production‑ready LakeFlow pipeline, minus the usual human bottlenecks.
Meet the Agents (and their Olympian counterparts)
Zeus - The Overseer
As the king of the gods, Zeus sets the overall direction. In Pantheon, he acts as the orchestrator: understanding the user’s intent, coordinating the agents, and knowing when to move forward, loop back, or pause for human judgement.
Athena - The Designer
Athena is the goddess of strategy and wisdom, and her Pantheon counterpart is responsible for designing model layers and shaping the transformation logic. She takes the profiled data and defines a clean, structured approach from landing through to silver.
Hermes - The Custom SQL Specialist
Hermes is the swift messenger, but also the god of craftsmanship and transitions. In Pantheon, he injects or adapts SQL logic where bespoke requirements arise, bridging the gap between standardised patterns and edge cases.
Hephaestus - The Builder
The master craftsman of Olympus. Here, Hephaestus assembles the final pipelines, constructs the LakeForge‑standard artifacts, and packages everything with the right documentation and lineage. His outputs are built to last.
Artemis - The Validator
Known for precision and guardianship, Artemis ensures nothing slips through. Her role is to apply semantic checks, validate logic, and enforce LakeForge governance standards before anything moves downstream. She’s the quality gate that keeps everything production‑worthy - the huntress of bugs.
Together, these agents act like a finely tuned assembly line: fast, disciplined, and predictable - without losing the intelligence or craft behind the scenes.
What Pantheon Is Not
As powerful as Pantheon is, it isn’t magic, and it isn’t a black box.
- It doesn’t bypass governance; it enforces it.
- It doesn’t generate fragile “AI‑magic” code; it produces patterns grounded in LakeForge.
- It doesn’t lock you in; the outputs are yours - fully auditable, modifiable, and aligned with open best practices.
Pantheon delivers outcomes, not shortcuts.
Conclusion
Pantheon and LakeForge were created side‑by‑side, each shaping the other as we worked out what modern, reliable data engineering should feel like in this new vibe-coding world. LakeForge provides the structure and standards, and Pantheon brings them to life through its team of specialist agents. The result is an ecosystem built from day one to make high‑quality engineering faster, clearer and far more enjoyable to deliver.
If you want to find out more about LakeForge, download the flyer here, and feel free to reach out for a demo.
