Databricks and Neon | Databricks Blog

by oqtey
Databricks and Neon | Databricks Blog

Today, we are excited to announce that we have agreed to acquire Neon, a developer-first, serverless Postgres company. Neon’s co-founders are among the very few people in the world who could re-architect Postgres with true separation of storage and compute, built for modern developers and AI systems. Their world-class team of Postgres hackers and database veterans will join Databricks to deliver serverless Postgres at production scale to developers in an AI-native world.

A developer-first mission, born from Postgres expertise

Four years ago, the Neon co-founders joined together with a vision to disrupt the database industry. They observed that the foundations of database technologies were largely designed for the 90s era. Their goal was to build a new database platform that would dramatically improve experiences for developers, in some fundamental ways:

  1. They foresaw that Postgres would become the de facto standard for databases and had the vision to create a serverless Postgres platform.
  2. They were determined to make it possible to create a new Postgres instance in seconds, so developers wouldn’t need to wait. 
  3. They set out to simplify the operational aspect of database scaling by automating it as load changed, so developers could start very small and not worry about over- or under-provisioning.
  4. They wanted to enable rapid experimentation and testing by supporting instant forking and branching of databases and creating fully isolated databases, as databases are often one of the most difficult modules to test in end-to-end application testing.

A few years in, the Neon team engineered a new, innovative database architecture that decouples storage scaling from compute scaling, which ultimately enabled all of the above goals. When Neon first launched, developers raved about the speed, the simplicity, and the ability to branch and fork their databases like Git does for code.

The shift toward AI agents

As Neon became GA last year, they noticed an interesting stat: 30% of the databases were created by AI agents, not humans. When they looked at their stats again recently, the number went from 30% to over 80%. That is, AI agents were creating 4 times more databases versus humans.

If you think of AI agents as your own massive team of high-speed junior developers (potentially “mentored” by senior developers), it’s then not surprising that the same capabilities the Neon team focused on that made Neon great for developers also made Neon great for these AI agents:

  1. Postgres open source ecosystem: All frontier LLMs have been trained on the vast amount of public information available about the Postgres open source ecosystem, so all AI agents are experts in using Neon, which is built on Postgres.
  2. Speed: Traditional databases were designed for humans to provision and operate. It was OK to take minutes to spin up a database. Given AI agents operate at machine speed, ultra rapid provisioning time becomes critical.
  3. Elastic scaling and pricing: The decoupled storage from compute serverless architecture enables extremely low-cost Postgres instances. It’s now possible to launch thousands or even millions of agents with their own databases cost-effectively.
  4. Branching and forking: AI agents can be non-deterministic, and “vibes” need to be checked and verified. Neon’s ability to instantly create a full copy of a database, not only for schema but also for the data, allows all these AI agents to be operating on their own isolated database instance in high fidelity for experimentation and validation.

Shared DNA

We have long known Nikita Shamgunov, Heikki Linnakangas, and Stas Kelvich – the founders of Neon. Before Neon, Nikita started his career working on SQL Server and later became well-known in the database community as a co-founder and CEO of SingleStore, driving the startup’s business to over $100M in annual revenue. Heikki has been a committer on Postgres for two decades – he started his Postgres journey when he was bored on paternity leave and decided to look into Postgres internals for fun. Stas is an ex-physicist who got into Postgres hacking because he needed a better R-tree implementation for his tourism search engine startup.

Databricks and Neon share the same DNA in building hardcore technical innovations at the infrastructure layer. We also share the belief in the importance of open source: we originally started the Apache Spark™ project at UC Berkeley, which also happens to be the original birthplace of Postgres, the open source database project Neon builds on.

Looking forward

OLTP databases represent a $100B market dominated by products that were built decades ago. We believe it is time for this market to be disrupted by developers and AI agents. Together with the Neon team, we look forward to building the most developer and AI agent friendly database platform.

What does this mean for existing Neon customers and partners? We are fully committed to the future of the Neon platform. The roadmap remains ambitious, and we intend to continue making Neon the best database platform for developers. After closing, existing customers and partners can expect continued support and innovation – now with the full backing and resources of Databricks. It’s a great moment for existing Neon users, and an exciting one for developers just discovering Neon. Go launch a new Postgres instance on Neon today!

We’re equally excited about what this will mean for Databricks’ enterprise customers. We plan to share much more at the upcoming Data + AI Summit in San Francisco, June 9–12.

Check out Neon’s blog to hear directly from them about why they’re excited to join forces.

Related Posts

Leave a Comment