Introduction
I use Claude every day. I use it for writing, coding, debugging, researching, sorting out messy thoughts, making work move faster when my brain has had enough (I even get it to help me with parenting!).
So yes, I get the excitement around Anthropic.
Claude is strong. Claude Code is genuinely useful. MCP is a great invention by the Anthropic team. Skills are useful. The Agent SDK is moving fast. The whole ecosystem feels like it is becoming more practical by the week. But lately, I keep seeing this in boardrooms:
“Claude is really useful for me.”
And then making the leap to:
“Claude is ready to run across the business.”
That leap is where things get risky. Useful to one person is not the same as safe, governed and auditable across an organisation.
When I use Claude, I know the context in my head. I know what I meant. I know which files I have used. I can spot when it is confidently wrong. I know when to challenge it. A regulated organisation cannot run on that.
If Claude is touching claims, credit, pricing, customer records, product data, underwriting, reporting or operational workflows, the question is not “is the model good?”
It is:
- Can we prove what it saw?
- Can we prove what it did?
- Can we prove who approved it?
- Can we prove it was allowed to use that data?
That is where Databricks becomes integral, and here are 10 reasons why.
1. You need to prove which data the AI used
Databricks feature: Unity Catalog, Lineage
"The model said so" is not an answer a regulator accepts. When Claude drafts a credit memo or supports an underwriting decision, someone will eventually ask which datasets, documents, prompts and business rules went into it, and whether those were the right sources of truth. You need to be able to show that, and to trace how the data became context, how the context became an output, and how the output fed a human decision.
2. You need access controls that survive contact with reality
Databricks feature: Unity Catalog, row filters, column masks, on-behalf-of-user authorisation
AI doesn't fix your permissions problem. It makes it more dangerous. If someone can't reach sensitive customer data directly, they shouldn't be able to reach it through a well-written AI summary instead.
This is also where most agent deployments quietly go wrong. By default an agent runs under its own service identity, which usually means broad access and a layer of prompt-level filtering you are trusting to hold. On-behalf-of-user (OBO) authorisation changes that: the agent runs with the permissions of the person using it, so Unity Catalog applies the same row and column limits it would if that person queried the data directly. Enforcement moves off the prompt, where it's fragile, and onto the platform, where it isn't. Without it, you haven't built an assistant. You've built a very polite data leak.
3. You need an audit trail
Databricks feature: MLflow Tracing, system tables and audit logs
Who asked, what they asked, what was retrieved, what came back, which workflow fired, whether a human signed off. Audit trails are boring right up until something goes wrong, at which point they become the only thing anyone cares about. Claude produces the output. You still have to prove the steps it took to get there.
4. From output to action
Databricks feature: Unity Catalog Lineage, MlFlow, Lakeflow Jobs
Knowing where the data came from is only half of it. The other half is what happened next. Once an agent stops answering questions and starts drafting credit packs, triaging complaints, updating CRM records or preparing a regulatory submission, you need to trace how context became an output, how that output became a human decision, and how that decision became a system action. Not lineage in the tidy architecture-diagram sense. Lineage in the "can we defend this when someone important asks" sense.
5. Somewhere to keep the evidence
Databricks feature: Unity Catalog, MLflow Model Registry and Lakehouse Monitoring
Not every use case is high-risk. But plenty of regulated ones sit close enough to sensitive decisions that governance can't be an afterthought. The sane version of this is one place that holds your approved use cases, lineage, access policies, model and prompt versions, evaluation results and review points. A shared folder full of governance documents named FINAL_v9 is not an operating model.
6. You need to stop bad data becoming confident nonsense
Databricks feature: Lakeflow, Delta Lake
RAG over messy data is still messy data. Duplicate records, inconsistent product terms, stale policy documents, claims rules scattered across systems. Even the most sophisticated model won't fix any of that. It will summarise it beautifully, which is worse, because now the mess sounds authoritative. Feed it rubbish and you get articulate rubbish.
7. You need model evaluation, not vibes
Databricks feature: MLflow, Agent Evaluation, Lakehouse Monitoring
A demo looks great with five friendly prompts and a room that wants it to work. Production is where you find out whether answers are grounded, whether retrieval held up, whether permissions were respected, and whether a prompt change quietly broke something. And you have to keep checking, because models, prompts, data and regulations all keep moving.
8. You need monitoring from insight to action
Databricks feature: Lakehouse Monitoring, MLflow, Agent Evaluation
Passing the pre-launch tests is not the finish line. Models change, prompts change, data changes, regulations change, and people will find creative new ways to misuse anything you hand them. So you need to catch it when retrieval quality slips, hallucination rates climb, latency starts hurting the workflow, costs spike, or a use case quietly wanders past the boundary it was approved for. The demo is meant to look easy. Production is where the bodies are buried.
9. You need one foundation, not another zoo of pilots
Databricks feature: Vector Search, Feature Store, Unity Catalog, Unity AI Gateway and MLflow
Most large companies already have too many AI experiments. A chatbot here, a document assistant there, someone in innovation wiring a model to SharePoint and giving it an adorable name which is almost always a warning sign. Six months later nobody knows which version of the truth each one is running on. Lots of interesting creatures, questionable containment strategy. Reusable, governed assets are how you close the zoo: shared corpora, retrieval indexes, features, prompts, models, evaluation sets. That's how AI becomes infrastructure rather than a drawer full of clever prototypes.
10. You need security and procurement to say yes (Easily!)
Databricks feature: Unity AI Gateway, Unity Catalog and serving endpoints
Enterprise AI doesn't scale because one team found a model they liked. It scales when security, legal, risk, compliance, data, architecture and procurement can all sign off without needing a lie down, clear on where the data sits, how access is controlled, how outputs are monitored, how costs are governed, and who owns the thing once the excitement wears off. A control layer that answers those questions is what turns "great demo" into "approved for production." Deeply unsexy. Also, more or less, the entire ballgame.
They were built to sit together
None of this is Databricks versus Anthropic. The two signed a multi-year partnership in 2025 to bring Claude natively into the platform, so you can point it straight at your own governed data. Anthropic builds the reasoning. The platform governs the data underneath. They were designed to work together, which is rather the whole point of everything above.
Models change. The data doesn't.
Next month Anthropic will ship a better model. Next year the agent platform will be more capable again. Good. That's the point.
But your data has no release cycle. The decisions your agents make this quarter, the trail they leave, the governance you did or didn't build, none of that improves when Claude 6 arrives. Every other part of the stack will turn over. The data and the controls around it are the part that compounds.
You can rent a model. You can rent a whole agent platform. You can't rent a governance layer. In a regulated industry that's not a technicality. It's the difference between AI as a productivity tool and AI the business can actually stand behind.
That's the work we do at Advancing Analytics: getting Anthropic's agents connected to a governed foundation - so the answers hold up when someone senior starts asking. If that's the problem on your desk at the moment, it's worth a conversation.
