Brickster Newsletter

How a chatbot became Databricks' home screen

Genie's quiet 2026 pivot changes the comparison set entirely.

Two years ago, Genie was Databricks' answer to natural-language SQL. A feature, not a strategy. This week, that changed. Genie can now reach beyond the lakehouse, pulling from Confluence, internal wikis, and docs. The Unity AI Gateway routes governance across foundation and on-platform models from one place. And the positioning has shifted entirely. Genie is no longer a feature you toggle on. It's the home screen business users open to start their day.

Two months ago, Genie's natural comparison was other natural-language SQL layers: dbt MetricFlow's chat, Snowflake's Cortex Analyst, Looker's bot. With this update, the comparison shifts to full BI tools: Looker, ThoughtSpot, Tableau Pulse. That's a much bigger fight, and a different game.

Genie's edge is that it's native to the lakehouse. The bet is that business users actually open it first, before any dashboard. If they do, the entire BI category gets pulled into Databricks' platform. If they don't, this becomes just another chatbot. Either way, it earns the version bump.

How seven UC Berkeley researchers built a data empire

In 2013, seven researchers from UC Berkeley's AMPLab (including Ali Ghodsi, Matei Zaharia, and Ion Stoica) founded Databricks. The plan was simple. Commercialize Apache Spark, the cluster-computing engine they'd built to chew through massive datasets fast.

It started as managed Spark. Then customers showed up with the same complaint: their data lakes were a mess, and their warehouses were too rigid for the work they actually wanted to do. Databricks' answer was a new architecture, the Data Lakehouse.

By 2026 they're rewriting themselves again, this time as the Data Intelligence Platform. GenAI baked into the lakehouse from the bottom up.

The Foundation

Databricks is founded to provide a cloud-based platform for Apache Spark.

Databricks Delta

A proprietary feature bringing ACID transactions and reliability to messy data lakes. Open-sourced as Delta Lake in April 2019.

MLflow

Launched to standardize the chaotic machine learning lifecycle.

The Lakehouse & SQL

The "Lakehouse" paradigm is born. Databricks SQL brings BI to the lake.

Unity Catalog

Centralized governance for data and AI assets across multi-cloud.

Data Intelligence Platform

MosaicML acquisition ($1.3B) and deep GenAI integration into the lakehouse.

Open Ecosystem

Open-sourcing Unity Catalog and acquiring Tabular to unify Delta Lake and Apache Iceberg.

Databricks Apps & AI Agents

Native app development on data, Agentic AI capabilities, and advanced serverless architecture.

Lakebase, Agent Bricks, and $5.4B

Lakebase reaches GA. Agent Bricks ships. Databricks One rebrands to Genie. Annual run-rate crosses $5.4B.

Tabular (2024 • >$1B)

Bridging the gap between Delta Lake and Apache Iceberg for unified data interoperability.

MosaicML (2023 • ~$1.3B)

A massive move into Generative AI, enabling enterprises to train and deploy custom LLMs securely.

Okera (2023)

Integrated into Unity Catalog to bolster AI-centric data security and governance policies.

AI Infrastructure

Enterprise AI scaling, in detail

Three deep-dive posts on the infra side of production-scale AI: Superhuman's 200K-QPS inference platform, monitoring 10 trillion samples a day, and a fundamental rethink of distributed systems for serverless reliability.

• How Superhuman and Databricks built a 200K QPS inference platform together · news
• 10 trillion samples a day: scaling beyond traditional monitoring infra at Databricks · news
• Rethinking distributed systems for serverless performance and reliability · news

Data Engineering

Lakeflow Connect's quiet expansion

Five new connectors landed: Outlook, Confluence, Meta Marketing, Jira, Zendesk. The bigger story under that is watermark-based incremental ingestion from SQL databases without needing Change Data Feed. A real quality-of-life win for a common pattern.

• Databricks News: watermark-based incremental ingestion, MCP in AI Gateway, Genie, Vector Search · video
• Ingestion without CDF · community
• Databricks is now supporting Microsoft Outlook in Lakeflow Connect (Beta) · community

Developer Tooling

CLI, SDK, and VS Code refresh

The toolchain shipped updates across the board. CLI v0.299.1 (unified hosts and workspace overrides), VS Code extension v2.10.7 (initial remote-dev compatibility), and a fresh push of OSS labs projects.

• databricks/cli v0.299.1 · release
• databricks-vscode v2.10.7 · release
• databricks-solutions/ai-dev-kit · project

1,232 DAIS talks, now searchable

Six years of Data + AI Summit, indexed and queryable from any AI client.

Every Data + AI Summit since 2020 has produced hundreds of talks. Keynotes, customer case studies, deep-dive breakouts, technical demos. Until now, finding a specific topic across them meant scrolling through 11 separate YouTube playlists.

This week, FlutterBricks indexed all of them: every talk tagged by year and conference, fully searchable through the on-site assistant or any MCP-aware AI client (Claude Desktop, Cursor, Codex, Continue). Ask "show me Photon vectorization deep-dives from DAIS '24" and you get them.

🤖 Stay ahead of the Databricks curve

Use the Brickster Assistant to instantly search releases, docs, and news.