Databricks News: Catalog and External locations in DABS, Schema Evolution, File Events, Queries Tags
Summary
Databricks Runtime 18.1 introduces schema evolution for inserts, managed file events for Autoloader, and a simplified `TABLE` syntax for querying. The video also demonstrates new features like the AI Gateway for LLM governance, query tags for tracking, and the GA release of the supervisor agent.
Summary from the captured archive. Follow the original source for full context.
More from databricks MVP Hubert Dudek
NewsDatabricks Breaking News: Lakeflow Jobs, DABs, Apps, AI, Private Network Gateway
The video covers multiple new updates in Databricks, including Lakeflow connectors, task value loops, state repartitioning, DAB enhancements, SQL functions, serverless version 6, Unity Catalog skills, app spaces, and network gateways. It demonstrates how to configure Lakeflow job tasks, implement task value loops, and manage serverless compute permissions and environments.
Newsdatabricks TaskValue and Lakeflow - orchestrate using Python lists and arrays
NewsDatabricks News: ZeroOps, DABs, Indexes, Genie, sandboxes, migration from PowerBI, secrets
Zero Ops automatically detects errors in jobs and data quality with lineage analysis and proposes code fixes, while DABs now default to direct mode instead of Terraform with automatic state migration. Full-text search indexes deliver 400x faster queries on billion-row tables, Genie automatically converts PowerBI dashboards to Databricks metric views, and Unity Catalog secrets support granular read and reference-only permissions.