Model Serving
Recent items mentioning Model Serving across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks Apps now support Model Serving endpoints as native Unity Catalog securable resources through updates to the Python and Java SDKs 23, complemented by expanded Terraform exporter dependency support for serving endpoints 6. In production architectures, Databricks published patterns for querying managed serving endpoints directly from SQL using ai_query 8 and coupling them with Lakebase and AI Search for low-latency recommendation scoring 7.
Generated daily from the 9 most recent items mentioning Model Serving. Click any [N] to jump to the source.
Databricks Apps vs Model Serving: Authentication, Cost, and Performance Compared
The Databricks Java SDK now includes the MODEL_SERVICE enum value for Unity Catalog securable types in Databricks Apps. This update allows developers to configure and reference model serving endpoints as securable resources within app manifests.
The Python SDK now includes a new model_service enum value for Unity Catalog securable types under Databricks Apps. This update allows practitioners to configure Model Serving endpoints as securable resources within Databricks App manifests.
From post-training to production: How TinyFish runs its web-native model on Databricks AI Runtime
TinyFish established a continuous path from post-training to deployment by using Databricks AI Runtime on managed GPUs, MLflow for evaluation, and Databricks Model Serving for production inference. Tested across 2,178 web-agent tasks, the resulting web-native model achieved a 77.6% task-success rate in internal go/no-go benchmarking.
Use Genie Code outside the workspace
Hey! I’m a Databricks product manager working on third-party development experiences. We just shipped Genie Code CLI , a new coding agent specialised for data and AI work. We’d love for you to try it out and share what you want to see next! Why we built this: Many of you prefer working locally with agents, but these agents might not always know the best way to work with Databricks. We wanted to provide an agent preconfigured to understand your Databricks data, pipelines, models, and apps. Use this when you want: Minimal setup and maintenance to use an agent that works effectively with Databricks. Governance provided through Unity Gateway. Access to agents outside of the Databricks workspace without model subscriptions or contracts. Getting started: (1) Install the latest version of the Databricks CLI . (2) Install Genie Code CLI : macOS, Linux, WSL: curl -fsSL https://github.com/databricks/genie-code-cli/releases/latest/download/install.sh | bash Windows powershell: powershell -ExecutionPolicy Bypass -c "irm https://github.com/databricks/genie-code-cli/releases/latest/download/install.ps1 | iex" (3) Start an interactive session by running genie in your project directory. Note that usage is billed through Unity Gateway’s model serving capabilities . For full details, check out the Genie Code CLI docs . Share your feedback: We’re releasing this fast to get your input asap. Please leave feedback and questions in the comments or reach out directly with a Reddit message! submitted by /u/BricksTrixTwix [link] [comments]
This release adds resources and data sources for private network gateways alongside Mason sessions and managed memory components. The resource exporter expands dependency support to include Lakeflow pipeline settings, model serving endpoints, Databricks Apps, cross-cloud cluster policies, and several new job task types.
Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks
A complete reference architecture is now available for building a real-time, multi-stage e-commerce recommendation engine on Databricks using AI Search for candidate retrieval, Lakebase for online feature serving, and Model Serving for low-latency inference. By unifying batch precomputation and real-time scoring on a single governed platform, this design replaces fragmented ML stacks to eliminate glue code and deliver end-to-end lineage from raw clickstream to production predictions.
Running open-Jev in SQL on Databricks
Run models like SemIf-OpenJev directly on governed Databricks data using serverless GPUs and managed Model Serving. Query custom endpoints from SQL or Lakeflow jobs using ai_query to return structured decisions and probability scores at scale, with an importable notebook to deploy and adapt the workflow.
NewsHow adidas Uses Databricks to Build Better Products
Adidas uses Databricks' lakehouse platform to centralize all its data—from product to football-related insights—enabling faster analytics across the organization. The company's Genie analytics tool helps analysts spend less time processing data and more time on strategic questions, ultimately supporting better product development.
Multi-region model serving on Databricks with OpenSharing
MLflow 3.16.1
MLflow 3.16.1 removes the default basic-auth admin password from basic_auth.ini as a security fix (GHSA-gq3w-7jj3-x7gr) and adds support for span links in Unity Catalog traces. The release also fixes trace location resolution in Databricks Model Serving, includes a timeout option for the @scorer decorator, and improves web UI access control for non-admin users.
Release introduces three new workspace services (AI Functions, Domains, Sandbox), extends Feature Engineering with new backfill and operation management methods, and adds AWS Secrets Manager and Azure Key Vault connection support. Multiple breaking changes remove fields from model serving and catalog configurations that may require code updates.
MLflow 3.16.0
MLflow 3.16.0 makes the redesigned trace explorer the default interface, introducing natural-language custom trace views via the MLflow Assistant, session grouping for multi-turn conversations, and span links. The update also adds Unity Catalog model service support for built-in evaluation judges, per-user AI Gateway budget policies, and fail-closed authorization by default.
The SDK adds an execute_command_sync() method for sandbox and new fields for ML features (mode, time_window, full_feature_name). Breaking changes remove routing and rate limiting configuration fields from model serving APIs.
Model Serving An internal error occurred during feature store lookup all deploys failing
NewsHow AI and Data Keep 2.3 Million Lawns Healthy | TruGreen & Databricks
TruGreen uses Databricks Genie and Lakehouse to manage 2.3 million lawns with AI that optimizes service timing and predicts customer churn using weather, soil, and service data. The system enables non-technical branch managers to take daily actions through customized reports without requiring data expertise.
The CLI adds databricks environments setup-local to provision matched Python environments for Databricks compute targets and extends aitools install to support Gemini CLI and Pi. Bundles fix the ignored bundle.deployment.lock.force setting, add pipeline cascade_on_destroy control, improve experimental job_runs with idempotency tokens and completion waiting, and add UC secrets resource support.
NewsHow FOX Sports Uses AI to Power Search
Fox Sports rebuilt their search system on Databricks to handle rapidly changing sports information by continuously streaming player, team, and content data into the index while computing real-time trends. The system uses semantic vector search with time-weighted ranking to surface fresh content higher, doubling the rate at which users find what they're looking for.
TutorialsDetect Energy Theft Faster with Genie
Databricks demonstrates an end-to-end AI application that detects energy theft, automates investigations, and generates executive reports using Unity Catalog and Genie. The video walks through an architecture featuring Lakebase for transactional storage, model serving for machine learning and LLMs, and AI gateways for governance and cost control.
This release adds effective entitlements to workspace assignment details and serverless compute ID support for job clusters. It also updates model serving telemetry configurations to include fields for table names and telemetry profile IDs.
What happens in the milliseconds after you tap pay
This sample Databricks App demonstrates how to achieve low-latency real-time fraud scoring by pairing route-optimized Model Serving with Lakebase Postgres for online feature lookups. Under load testing of 5,000 requests, this architecture achieved end-to-end latencies of 27 ms at p50 and 37 ms at p95 while maintaining a 100% success rate.
The SDK now supports specifying a parent path for jobs and associating a Git credential ID with workspace repositories. Model serving configurations now include CpuLarge and CpuMedium options for workload types.
The Java SDK adds a parent path field for job creation and settings, along with git credential ID support for workspace repositories. It also introduces CPU_MEDIUM and CPU_LARGE workload type enums for Model Serving.
Databricks AI Model Serving in production: scaling, cost, and latency lessons
SSH connect adds --base-environment for custom base environments, and aitools install now uses plugins instead of raw skills. Bundle deployments fix drift on model serving endpoints and failed migrations on permissioned resources.
A breaking fix ensures query parameters specified in ForceSendFields are now explicitly sent over the wire even when holding zero values. Additionally, new API updates add support for Genie conversation visualizations, telemetry configuration on model serving endpoints, and expanded Postgres service options.
Databricks and NVIDIA: Building for the Agentic Era
Databricks and NVIDIA are expanding their collaboration to deliver an end-to-end AI platform, accelerating model training, inference, and agentic AI development on governed enterprise data. This includes multinode training in AI Runtime, GPU support in Databricks Free Edition, Model Serving Enhancements, and support for NVIDIA Agent Toolkit and industry-specific AI frameworks.
What’s New in the AI Platform: Agents for ML Engineering, Our Deep Learning Platform, and New Capabilities for Real-Time ML
Databricks shipped Genie Code, a coding agent for ML engineering, and AI Runtime, a serverless GPU platform for deep learning. Power real-time ML at scale with new Feature Store and Model Serving capabilities, including streaming features and high-QPS serving.
How ERGO Hestia reduced time-to-market with Lakebase and Mosaic AI Model Serving
ERGO Hestia modernized its real-time pricing engine with Databricks Lakebase and Mosaic AI Model Serving, reducing time-to-market by unifying data, features, and decisions for millisecond pricing. This eliminated extraction overhead and fragmented governance from their previous multi-hop architecture, enabling faster model deployment and instant market response.
Azure OpenAI v1 API support for External Model Serving / Mosaic AI Gateway?
Databricks Model Serving: The Complete Guide to Production ML at Scale
NewsBanks' Secret Weapon Against Money Laundering: Multi-Agent AI
Databricks demonstrates a multi-agent AI solution for Anti-Money Laundering (AML) operations, significantly reducing false positives and accelerating investigation cycles from hours to minutes. The platform unifies siloed systems, employs specialized AI agents for analysis and recommendations, and offers AI-assisted SAR generation and executive-level reporting with natural language chat.
NewsDatabricks Apps vs Model Serving: Authentication, Cost, and Performance Compared
Databricks Apps are now the recommended first choice for deploying agents due to their flexibility in handling full-stack applications with multiple components, offering faster iteration and local testing compared to Model Serving. Model Serving remains suitable for use cases prioritizing high QPS, governance features like AI Gateway, inference tables, and guardrails, or when scaling to zero is acceptable for cost optimization.
MLflow 3.11.1 introduces AI-powered issue detection in traces, AI Gateway budget alerts and spending controls, trace graph visualization, native Databricks gateway provider, and pickle-free model serialization. TypeScript SDK packages are now @mlflow-scoped and LiteLLM is no longer required for GenAI evaluation.
Databricks SDK for Python now supports new fields for defining ingestion pipelines, including connector type, data staging options, and detailed ingestion source information. External function requests in model serving can now specify a sub-domain.
NewsLakehouses: The Best Start to Your Graph Data and Analytics Journey
The video demonstrates how to build and deploy graph neural networks on a lake house architecture using Databricks instead of relying solely on graph databases. It showcases an end to end data science workflow including data ingestion, graph machine learning training with DGL and PyTorch, hyperparameter tuning with MLflow, and model serving via Unity Catalog.
NewsLLMOps: Everything You Need to Know to Manage LLMs
Databricks demonstrated how to build, evaluate, and productionize a retrieval-augmented generation question-and-answer bot using its Lake House AI platform. The presentation walked through setting up vector search, prompt engineering, model serving, and lake house monitoring.
NewsBuilding a Data Science as a Service Platform in Azure with Databricks
Building a data science as a service platform with Azure and Databricks involves identifying various user personas across an organization and then using tools like AutoML and MLflow to enable them. The video demonstrates how Databricks provides AutoML, managed MLflow, and MLflow model serving to support different user personas and enable them to build and deploy machine learning models.
EventsScaling AI At H&M
H&M continues evolving its AI platform to democratize and accelerate AI across the group, covering speed to production, data abstraction, feature store and pipeline orchestration. Its reference architecture, used by multiple product teams managing hundreds of models, supports interactive model development plus large-scale training and serving pipelines with full traceability, grounded in MLOps principles.
NewsKFServing, Model Monitoring with Apache Spark and a Feature Store
An open-source platform serves and monitors models at scale using KFServing, the Hopsworks Online Feature Store for enriching feature vectors, and Spark and Spark Streaming for production monitoring. Spark Streaming compares live traffic statistics against training distributions from the Feature Store to detect data drift continuously and raise alerts, shown in a live demo.
NewsModel Monitoring at Scale with Apache Spark and Verta
Verta presents a model monitoring framework that works unchanged in batch and real-time inference, applied to ML inference workflows built on Apache Spark and Databricks. It explains how ML monitoring differs from application performance and data monitoring, then demonstrates using the Spark engine to monitor batch model performance and identify serving pipeline errors.
NewsProductionzing ML Model Using MLflow Model Serving
MLflow Model Serving offers one-click deployment of models for real-time inference, with deployed versions managed through the MLflow Model Registry. The demo covers deploying and validating different model versions, consuming a served model from Power BI to build a prediction report, and managing access rights and monitoring.
NewsBuilding an ML Platform with Ray and MLflow
Ray and MLflow, two open source projects, combine to add scaling and experiment management to an ML platform. The talk introduces Ray Tune for distributed hyperparameter tuning and Ray Serve for scalable model serving, shows MLflow integrations for tracking and deployment, and demos a platform built on these tools.
NewsMLflow Model Serving
MLflow offers several ways to serve models, both in open source MLflow and in Databricks managed MLflow. The talk explains the basic differences between batch and real-time scoring, with special emphasis on the new Databricks production-ready model serving.
EventsTaking Machine Learning to Production with New Features in MLflow | Keynote Data + AI Summit EU 2020
Matei Zaharia presents new MLflow features for productionizing machine learning, including the Model Registry for model management and review, CI/CD APIs, model schemas that catch data format mismatches and integration with explainability tools. The keynote includes a CI/CD and MLOps demo by Kasey Uhlenhuth and Facebook's Lin Qiao on integrating PyTorch with MLflow.
NewsRay: Enterprise-Grade, Distributed Python
Ray, the open source distributed framework from UC Berkeley's RISELab, scales Python applications from a laptop to a cluster with low-latency task scheduling, distributed state management, near-linear scaling and an intuitive API. The talk shows how Ray works with Spark to build scalable enterprise data applications and when to choose Ray over alternatives.
NewsBest Practices for Engineering Production-Ready Software with Apache Spark
Montevideo Labs shares best practices for taking Spark prototypes to production: making code readable, debuggable, reusable and testable, architecting components for batch ETL, low-latency services and model serving, and packaging and deploying to the cloud. Live demos show how Spark's laziness lets code follow sound software patterns while Spark-specific logic stays separate from business rules.
TutorialsAutoML Toolkit – Deep Dive
The AutoML Toolkit from Databricks Labs automates and accelerates repetitive machine learning work, from feature engineering, culling and importance-based selection to model selection and tuning. A deep dive covers model serving and deployment, model documentation with MLflow, and inference and scoring.