Databricks Data & AI Summit Announcements 2025

Terry McCann

16 June, 2025

We’re back from a packed week in San Francisco – and what a week it was.

Last week saw the return of the Data + AI Summit, Databricks’ flagship conference that brings together developers, Spark nerds, partners, and Bricksters from all over the world 🌍. Once a year, we all descend on San Francisco to learn, share, and get a feel for where the Databricks platform is heading.

Before the event, I told quite a few people not to expect much in the way of announcements – purely because of how fast Databricks has already been moving in 2025. Turns out I was wrong. Very wrong. There were a number of genuinely brilliant announcements – the sort that could have a real impact on your data platform 🚀. The pace at Databricks isn’t slowing down – if anything, it’s accelerating 💨. In this blog, I’ll break down each of the major announcements, give you links to dig into the detail, and hopefully help you decide what matters for your platform.

If anything grabs your attention or you want to talk it through, just drop us a message – we’d love to help you figure out what it means for your business. This was also the first year Advancing Analytics sponsored the event 🎉, and we had loads of great conversations – we’d love to keep those going. What follows is a ranked list of the announcements – biased, of course (written by me, Terry McCann), but hopefully grounded in good reasoning and experience.

Let’s get into it 👇 I have also graded how excited I am by the announcement. 

1. Databricks One 🧱✨ (11/10)

If I had a dollar for every time I’ve said, “No exec wants to log into Databricks,” I’d be writing this from my yacht. This has been one of the most consistent bits of feedback I’ve had over the last few years. Execs don’t want a full-blown workspace – they want clean, curated, and easy. Databricks One is that. It’s a simplified, user-friendly interface, purpose-built for execs and non-technical users. No jumping through workspaces, no clutter – just direct access to the insights they care about.

As more of us build out BI dashboards, embed GenAI, and roll out tools like AI/BI and Genie, this kind of persona-driven UI becomes critical. Databricks One will abstract away the complexity and presents users with just what they need. It’s a UX tailored for roles, not roles forced to adapt to the UX.

I genuinely think this is a game-changer 💥. Right now, a lot of enterprise architectures rely on tools like Power BI, Collibra, or DataHub to plug gaps in the Databricks experience – especially when it comes to delivering to end users. But Databricks One starts to close those gaps. In doing so, it introduces a real threat to traditional dashboarding tools like Power BI and Tableau, especially once you layer in AI-native features. In short – this is a big win. A serious levelling up of the Databricks experience.

Read more about it here: https://www.databricks.com/blog/introducing-databricks-one

Is it available? No. Beta "later in the summer" expect Private Preview much later. This one will be a while before we have a working GA version on the iPad of your CEO. 

2. Lakeflow Designer 🌊🧠 (10/10)

This one follows nicely on from Databricks One – and we’re just as excited about it. Historically, Databricks has been a platform built by and for pro coders - those happy to dive into a notebook, drop into an IDE, and write Python, SQL or Scala until the job’s done. Databricks is a fantastic product for pro code.

But the world’s changing. There’s a growing pressure on skills, and tools like Matillion, Prophecy, and Alteryx continue to build popularity by offering a more accessible experience. They give people a way to build on top of Databricks without needing to be a Spark expert. This does come with limitations, but a less experienced user can accept those limitations for the development improvements. 

LakeFlow Designer introduces a visual, interface-driven way to build Spark Declarative Pipelines 🛠️. It brings the platform closer to the low-code/no-code experience that’s been missing. If I were running one of those other companies right now, I’d be more than a bit concerned. So if you're leaning towards Microsoft Fabric purely because of its UI/UX, I’d urge you to take a renewed look at Databricks through the lens of LakeFlow Designer. 

Read more about it here: https://www.databricks.com/blog/announcing-lakeflow-designer-no-code-etl

Is it available? No. Private Preview coming soon ("in the coming months"). This one needs to come quickly if it hopes to fight off those early Fabric adopters. 

3. Unity Catalog Metrics 📊🧠 (8/10)

Unity Catalog has always been Databricks’ flagship play for data governance. When they announced it was going open source last summer, the internet almost crashed – but since then, we’ve seen very little on what that actually means in practice. Still, it’s technically there if you want to dig in. This year’s announcement, was Unity Catalog Metrics – and this one hits a core pain point in how we build data platforms.

Here’s the problem: we try to push calculations as far back into the platform as we can. But once someone builds a KPI or metric definition in SQL or Spark, it often gets redefined – in BI tools, Excel exports, or duplicated pipelines. The result? Everyone’s talking about “margin” – but that might mean gross, net, EBITDA, or something else entirely. UC Metrics aims to fix that. It provides a central, governed interface to define metrics once and serve them consistently across BI tools. So not just consistent definitions – but actual metric serving 🔄. "Define once, use everywhere". We have seen other semanitic model parters do the same thing, and this looks to be another canabilisation of partners in to the product. 

Now, I’ll admit – I got swept up in the excitement. But there was one big gap I didn’t notice until later... All the talk was about integrations: Sigma, ThoughtSpot, Domo, Tableau... but no mention of Power BI or Microsoft Fabric. That’s a glaring omission in my view. I don’t know if Microsoft declined to take part or if support is coming later – but as it stands, if you’re a Power BI user, it looks like you’ll only see the metric definition in UC Metrics. No live serving. That’s a bit of a miss, and one we’ll need to keep a close eye on. Still – the concept is strong. And for orgs looking for clean, governed metrics across tools, this is a big step in the right direction.

Read more about it here: https://www.databricks.com/blog/whats-new-databricks-unity-catalog-data-ai-summit-2025

Is it available? Yes. In Public Preview - Just turn it on. 

4. Lakebase 🛶🧮 (7/10)

One of the bigger surprises this year was the announcement of Lakebase, coming off the back of Databricks’ acquisition of Neon – a managed Postgres provider.

Now, on the surface, it might sound like Databricks is just launching a managed Postgres service. But that’s not quite what’s happening. What Ali Ghodsi laid out is a bigger ambition: Lakebase is being pitched as a new architecture for OLTP workloads – something very different from what Databricks has traditionally focused on.

How new this actually is... well, that's debatable. There are clear similarities with how platforms like Azure SQL Data Warehouse – particularly in the separation of storage and compute. But traditionally, those platforms were built for OLAP, not OLTP. This is Databricks attempting to blend that architectural model with transactional use cases – all built on Postgres and open source foundations.

There are some standout features. Near-instant branching of data and code. Built-in support for AI agents that can anticipate load, spin up instances, and checkpoint state automatically. Full integration with the lakehouse. And for us at Advancing Analytics, it opens the door to reducing reliance on external systems for metadata or lookup services – making things a lot more natively Databricks.

That said, I do have a note of caution. Databricks has always played to its strengths – big data, AI, machine learning. Not traditional OLTP. This move could either signal a shift towards being a truly multimodal database platform – more Cosmos DB than Spark – or it could be a distraction. Chasing transactional workloads may end up pulling focus away from their core value prop.

Read more about it here: https://www.databricks.com/blog/what-is-a-lakebase

Is it available? Yes. In Public Preview - Just turn it on. Well wort of, the wording is a little confusing, we are launching a product in PP, but also a new paradigm... 

5. Databricks Apps 🧩⚙️(5/10)

Databricks Apps is now GA – meaning you can officially run apps and production workloads natively on top of the Databricks platform. That’s a big step forward. 

But... we’re still stuck with the same pain point we mentioned back in Databricks One: if someone wants to use an app in anger, they still need to be inside the workspace. Until there's a clean way to expose these apps outside of that workspace – ideally integrated directly into the Databricks One interface – we’re not quite there yet. That said, Databricks Apps + Lakebase is a pretty compelling combo. The idea of running a fully secure system that supports both OLTP and OLAP workloads, in a single interface, is seriously powerful 💪. It brings Databricks a lot closer to the kind of unified data experience the industry’s been chasing for years.

If they get the UX right, this could be a massive unlock.

Read more about it here: https://www.databricks.com/blog/announcing-general-availability-databricks-apps

Is it available? Yes. It is now GA. 

6. Lakeflow is now GA 🔁⚡(5/10)

Last year, Databricks introduced Lakeflow, aimed at tackling one of data engineering’s longest-running pain points: Change Data Capture (CDC). CDC has always been messy – a mix of custom connectors, external services, and flaky integrations. LakeFlow attempts to clean this up by providing managed ingestion from enterprise apps, databases, file systems, and real-time streams – all without needing to bolt on half a dozen external tools.

Now, to be clear, you don’t have to do it the Databricks way. But Databricks has definitely created a path of least resistance that nudges you into using Lakeflow Declarative Pipelines (what used to be DLT, now open source and rebranded - more on that soon).

The big news is that Lakeflow is now GA 🎉, with support for major connectors like Salesforce, Workday, Google Analytics, and SQL Server – making managed CDC on Databricks a lot more accessible. If you’ve been wrestling with CDC complexity, this is one to have a proper look at.

Read more about it here: https://www.databricks.com/product/data-engineering/lakeflow-connect

Is it available? Yes. It is now GA. 

7. Agent Bricks 🧱🤖 (Yes, that’s really the name)  (4/10)

I’ll be honest – I’m not totally sold on the name Agent Bricks. Sounds more like a sci-fi character than an enterprise AI feature. Is it chasing down Neo, or offering smarter AI ops? Time will tell.

What Databricks is actually tackling here is a real problem: building and scaling agentic systems is hard. Evaluating large language models is messy. Benchmarking is vague. Cost and quality optimisation is slow and painful. There's too much to configure, too many levers, and teams often end up stuck in long-winded tuning cycles just trying to get models into production.

Agent Bricks is Databricks’ attempt to simplify that by introducing a framework for:

  • Defining task-specific agents

  • Auto-generating evaluation pipelines

  • Auto-optimising for cost and quality

All powered by Agent Learning from Human Feedback (ALHF) 

You can think of it as a set of solution accelerators for use cases like:

  • Information extraction

  • Knowledge assistance

  • Multi-agent orchestration

  • Custom LLM workflows

It all starts with natural language: you describe the task, and Agent Bricks does the rest – evaluating and optimising in a loop. In theory, this could dramatically cut down the time and complexity of delivering GenAI workloads.

We’ve been solving similar problems with our own in-house tech, Gener8, built on top of Azure and Databricks. If you're exploring these kinds of use cases, drop us a line – happy to chat through what’s working in the real world.

One important caveat: Agent Bricks is still in beta, so don’t expect GA-ready maturity just yet.

Read more about it here: https://www.databricks.com/product/data-engineering/lakeflow-connect

Is it available? No. Not for a while yet. It is open for Beta testing.  

8. Spark Declarative Pipelines (DLT Goes Open Source)

(4/10) 

At last year’s summit, Databricks pulled a bit of a showstopper by open-sourcing Unity Catalog live on stage. This year, they went for the encore: the engine behind Delta Live Tables (DLT) is now open source.

Specifically, Spark Declarative Pipelines have been committed to Apache Spark 4.0. That’s a huge move – and a clear signal of Databricks doubling down on their “open by default” message. They're effectively donating DLT to the Apache ecosystem, giving the broader community access to something that’s been core to their managed pipeline experience.

It’ll take a bit of time before we see these pipelines fully functional in the open source Spark builds – but still, this is a major milestone. Declarative data engineering, in open source Spark? That’s big.

Read about it here: https://www.databricks.com/blog/bringing-declarative-pipelines-apache-spark-open-source-project

9. Databricks Free Edition 🆓💻 (2/10)

This one’s a bit of a curious announcement – because, let’s be honest, Databricks has always had a free edition. Anyone who's used the Community Edition over the past eight years will know it’s been the go-to for training, quick demos, and teaching people the basics of PySpark.

It wasn’t perfect – limited resources, patchy availability – but it did the job. We’ve used it plenty ourselves for training courses over the years.

What’s different now is that Databricks Free Edition seems to be getting more love, with training content on Databricks Academy now available for free to everyone. That’s a notable shift. Academy has typically been locked behind customer agreements or paywalls – so opening it up makes a lot of sense for growing adoption. A small change on paper, but it’ll make a big difference to how new users engage with the platform.

Read more here: https://www.databricks.com/blog/introducing-databricks-free-edition

10. LakeBridge 🔗🚀 (1/10)

Earlier in 2025, Databricks announced the acquisition of BladeBridge, a code migration tool designed to automate the move of enterprise data warehouse (EDW) workloads into Databricks SQL. The idea? Help customers migrate and modernise faster – get off legacy tech and into Databricks with less friction.

This isn’t just about convenience. LakeBridge gives Databricks a way to onboard workloads quickly, allowing customers to deprecate legacy systems faster while freeing up time to focus on delivering business value.

At Advancing Analytics, we’ve been working with migration tools since day one. In fact, we’ve built our own platform-agnostic suite called Hydr8, which you can read more about [here]. We know that code migration is only one part of the journey – and LakeBridge helps speed that up. But let’s be honest: lift and shift is rarely enough. Most migrations are driven not just by cost, but by the need for new capabilities. And when you just move code from A to B without modernising, you’re often only tackling a small part of the challenge. Hydr8 does a lot more than just lift & shift. 

That said, LakeBridge is absolutely a step in the right direction – just one that needs to be used carefully and as part of a wider modernisation strategy.

Other announcements: 

Loads of announcements. Get in touch if you want to know more about how these new features will change how you interact with Databricks. 

Topics Covered :

Events Databricks
author profile

Author

Terry McCann