Skip to content
All topics

Service Principal

Recent items mentioning Service Principal across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.

52 recent items18 releases6 videos28 community threads
What's happening in Service PrincipalAI synthesis · updated 4d ago

Databricks CLI v1.20.0 removed the Terraform deployment engine for Asset Bundles, so teams must move to the direct engine or pin to v1.19.x, and it adds group-level run-as in bundles and expanded authentication options for Apps 9. The Python and Java SDKs now let you set scopes on service principal OAuth secrets 67. Practitioners are also questioning whether running Lakeflow pipelines as a group sidesteps service principal gatekeeping 3, and report that service principals created by the Genie App builder don't appear in the Databricks UI 8.

Generated daily from the 9 most recent items mentioning Service Principal. Click any [N] to jump to the source.

Reddit

Power bi import gateway strategy

How are you guys that still use power bi via import mode designing your gateway authentication? We are new to databricks and have a lot of schemas that require different levels of AD access. Import mode is the company standard so no direct query sso auth. We are unsure how to design a scalable gateway strategy . Do we use a service principal or allow publishers to use their own credentials on the gateway? submitted by /u/curiouscsplayer [link] [comments]

00curiouscsplayer5d ago
Reddit

How do you manage, in a Databricks Asset Bundles-based CI/CD, the permissions and access resources that a bundle cannot declare?

We have a DABs-based CI/CD in a monorepo and want to split it: deploy the DABs from the data-code repo and keep the Terraform infrastructure in a separate repo. The original idea was that Terraform would change at a different pace and its deployments would be manual, with no automated dependency on DABs deploys. However, at work they’re now saying this infrastructure part should be tied to the DABs deployment and not be manual, and I’m unsure where it should live and how to automate it. We mainly run into four cases: Cross-boundary grants: our pipeline needs to read a table owned by another team, so we need to grant SELECT on that table/schema to our service principal. Postgres grants: a job needs to connect to a Lakebase database and requires a GRANT on a Postgres schema, table, or role. Share contents: we need to add or remove tables and views from a Delta Sharing share consumed by an external consumer. Notification destinations: a job or pipeline needs to send alerts to a notification destination (email, Slack, webhook) that the bundle cannot declare. Today, all of these live in Terraform outside the bundle, so every time the data code needs one of these changes, we have to coordinate a separate MR and deployment in addition to the bundle deploy. Where do you put these in your pipeline: inside the bundle, in a separate Terraform stack applied after the bundle deploy, in the platform repo, or another way? What issues have you run into with each approach? Tx so much submitted by /u/Appropriate_Tip_9580 [link] [comments]

00Appropriate_Tip_95801w ago
Reddit

Run Lakeflow pipelines as a group: governance win or a way around SP gatekeeping?

As of this month you can set a Lakeflow pipeline’s run-as identity to an account group. Tables the pipeline publishes become group-owned in UC. Two patterns I expect to see: 1. Routing around SP provisioning. In a lot of orgs, creating a service principal is process-locked by a platform or identity team. Tickets, approvals, secret rotation. With run-as group, a team that already has an account group with the right grants can skip all of that. The conversation moves from “who can create an SP” to “can we just use the DE group?”. 2. Lower-maturity teams leapfrogging SPs. Teams still running pipelines as individual users will likely jump straight to groups. That’s better than personal identities, since nothing breaks when someone leaves. But it means they may never adopt SPs at all, and “good enough” becomes the permanent pattern. Questions for the sub: If you run pipelines as SPs today, are you planning to switch any to groups? Platform/admin folks: are you going to restrict this, and how? How are you handling auditability when the identity is a group instead of a single SP? Is group run-as a legit middle step for teams coming off user identities, or a dead end? My take: SPs still belong in prod. But I’d bet most orgs end up with group run-as spreading through both of these doors, and won’t notice until an audit asks who ran what. submitted by /u/OkImprovement7010 [link] [comments]

00OkImprovement70101w ago
Databricks CommunityGenerative AI

Databricks Genie One - Managing past conversation initiated by Databricks service principal account

001w ago
Databricks CommunityData Engineering

Are PATs (user or service principal) accepted by Databricks Apps?

002w ago
Databricks CommunityGenie Hub

Service principals created by Genie App builder are not visible in Databricks UI

003w ago
Reddit

You can now run an SDP pipeline as a group

A pipeline's run-as identity can now be an account group. Updates run with the group's permissions, and the tables the pipeline publishes are owned by the group. What it is Every pipeline runs as some identity. That identity decides what the pipeline can read and write, and who owns the tables it creates. Until now you could pick a user or a service principal. You can now pick an account group too. A Databricks role is a group under the hood, so setting run-as to a group also lets a pipeline run as a role. Why you'd want it If a pipeline runs as one person, it depends on that person's account staying active. When they leave and the account is deactivated, the pipeline can stop running and the tables it published can become inaccessible to everyone else. Service principals already fix part of this. Groups go a step further: a whole team owns the pipeline, and it keeps working as people join and leave the team. No owner reassignment when someone moves on. How to use it https://preview.redd.it/1ml56gev61uh1.png?width=1848&format=png&auto=webp&s=40275cf04ac3a837108289df8e8f16a855acfd98 In the UI: Open the pipeline, click Settings. Click the pencil next to Run as. Pick an account group you belong to, or one you have Assume permission on. Groups show up next to users and service principals. Click Save. The new identity applies on the next update. You can also assume a group’s identity to change / create a pipeline. And you can change the owner of a standalone MV/ST to a group as well! { "name": "sales-etl", "run_as": { "group_name": "sales-eng" } } Declarative Automation Bundles: resources: pipelines: sales_etl: name: sales-etl run_as: group_name: sales-eng What you need CAN_MANAGE on the pipeline. Either membership in the group, or Assume permission on it (for groups used for exclusive access). The group has to be an account group assigned to the workspace. Workspace-local groups don't work, and neither do the built-in admins, users and account users groups. Grant the group every permission the pipeline needs. Updates run as the group, so your own access to the source and data isn't enough. Docs: https://docs.databricks.com/aws/en/ldp/configure-pipeline#set-the-run-as-identity-to-a-group Please let us know what you think! submitted by /u/saad-the-engineer [link] [comments]

00saad-the-engineer3w ago
Reddit

Azure Databricks, two-week-old account: partner pay-per-token endpoints (Claude, Grok) have never once succeeded — "Databricks-set rate limit of 0" — and the Assistant has "no daily token allowance". Open models work. What gates this?

Azure Databricks, Premium, one account with three workspaces, Unity Catalog, Azure Marketplace billing (no card, no trial credits left). The account was bootstrapped on a trial workspace about two weeks ago and moved to Premium six days later. I've spent two days on this and want a sanity check from anyone who has seen it. The facts, from system.serving.endpoint_usage: • Every open-weight pay-per-token endpoint has worked since the trial: gpt-oss-120b has ~50 successful calls going back to the trial period, Llama 3.3 70B likewise, zero refusals. • No partner pay-per-token endpoint has ever returned a success on this account. The very first call anyone made to databricks-claude-sonnet-5 was refused, and every call since, on every Claude endpoint and on Grok, in all three workspaces, for users and service principals alike: 403 PERMISSION_DENIED: The endpoint is temporarily disabled due to a Databricks-set rate limit of 0. • It is not our AI Gateway config: I removed every rate limit from one Claude endpoint and invoked it — same 403. The endpoints' config shows nothing else. • The notebook Assistant worked during the trial. On the paid account it refuses everyone, admins included: *"This workspace has no daily token allowance for the assistant."* Underneath, /ajax-api/2.0/conversation/llmproxy/ returns 429 {"type":"daily_token_limit_reached","daily_limit_tokens":0,"message":"…Daily token limit of 0 reached. Resets at midnight UTC. (DTB)"} and it recomputes to 0 every midnight. • Once, the Databricks Apps in the workspace were stopped by the platform with *"App compute was stopped due to workspace or account status"* while every workspace showed RUNNING. apps start brought them back. Red herring, for completeness: we also created a Unity AI Gateway per-user budget whose first version had a $0 threshold with BLOCK_USAGE. That blocked Genie for a day, exactly as documented, and was fixed. It was created a day *after* the first Claude refusal, so it isn't the cause of any of the above. What I've found: two Community threads describe "Databricks-set rate limit of 0" as a workspace trust-tier gate — trial-born accounts sit in TRIAL_VERIFIED, pay-per-token partner models are gated to PAYABLE_VERIFIED, a card alone doesn't flip it, only Databricks Sales/Support moves the tier. Both were AWS/personal accounts, neither resolved on-page. The account console's own API reports our account's feature_tier as STANDARD_W_SEC_TIER although the workspaces are Premium; I can't find what that field means. Questions • Does Azure Databricks with Marketplace billing go through the same PAYABLE_VERIFIED gate? Does it clear on its own after the first settled invoice, or does someone have to move it? • If you were moved: who did you contact (account team, or "Contact us" in the console) and how long did it take? • Is the Assistant's "daily token allowance" the same gate, or a trial allowance that simply goes to 0 when the trial ends on an unverified account? • Does feature_tier: STANDARD_W_SEC_TIER on the account object mean anything to anyone? submitted by /u/sumit671 [link] [comments]

00sumit6711mo ago
Reddit

Avoid Unity Catalog external location permission bugs by using managed storage

Hey, I work as a Databricks Engineer at Abilytics, so here's my take. If you are migrating existing tables to Unity Catalog, defining external locations incorrectly will break your grants. When you register an external location via CREATE EXTERNAL LOCATION, Unity Catalog validates your storage credential against the underlying cloud storage container. However, users often forget that read and write access on the external location is distinct from the IAM role or service principal trust policy. If a user has SELECT on a table, but lacks BROWSE on the external location, queries against parquet files directly via path-based access will fail with permission denied, even if the table-level grant succeeded. Always verify your storage credentials have explicit container-level policies attached before mapping external volumes. Furthermore, remember that path-based access like spark.read.load('s3://my-bucket/path') requires explicit external location grants, whereas managed tables abstract this entirely. Prefer managed tables default storage roots to bypass manual external location ACL synchronization issues entirely. Happy to go further — more on Databricks | Platform Engineering | AI at Abilytics , AI Systems, Data Engineering & Platform Engineering Services. submitted by /u/AbilyticsEng [link] [comments]

00AbilyticsEng1mo ago
Databricks CommunityData Engineering

Databricks now deletes unused service principal secrets after 90 days

001mo ago
Databricks CommunityLakebase Discussions

Lakebase Data API (GCP) returns jwk not found for valid service principal tokens

001mo ago
Reddit

Issues with oidc/v1/token endpoint (REST for zerobus-ingest)

Anyone else work with the zerobus-ingest? There is an auth-flow to get an "oauth token" from a service principal with an "oauth secret". In that flow, you are supposed to submit the authorization details. See docs: https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-kafka I had been requesting access (via authorization_details) to a table as shown in the example: "type": "unity_catalog_privileges", "privileges": ["SELECT"], "object_type": "TABLE", "object_full_path": "main.default.air_quality" ... but it wouldn't work. Was banging my head on it all day. The privileges granted to the service principal for this table were unrestricted " ALL PRIVILEGES ". It turns out that, in addition to "ALL PRIVILEGES", this "token" endpoint wanted me to grant "SELECT " to the service principal in UC governance as well. Else the authorization_details are not valid. This "token" api has a lot of sharp edges. Any chance there is a nuget client to wrap around it and protect us from these unfortunate nuances? Is it documented that ALL PRIVILEGES doesn't actually give us all privileges, if we request "SELECT" by name? submitted by /u/SmallAd3697 [link] [comments]

00SmallAd36971mo ago
Reddit

Ontology ranked the snippet. I wrote the gate I wanted before trusting a Genie space.

A commercial user asked what yield was last season. Two definitions live in the same lakehouse. One is plot-level, moisture-adjusted, borders out. The other is as-harvested, and a dashboard still uses it. Genie returned the second number. Not a hallucination. OntoRank did what it does: creator, usage, link to a certified asset, freshness. The loud dashboard won. Nobody had certified the measure the board pack is allowed to quote. That is the failure I care about. A green demo is not a release gate. What I actually require before that space faces a user who will paste the sentence into Slack: Golden questions written before the space exists. If you write them after you have tuned instructions, you are scoring an overfit prompt. Benchmarks are not instructions. Genie does not learn from the benchmark SQL. The number lives in a Metric View. A column comment and a knowledge-store formula are hints. Ontology will rank all three. Only the Metric View is a contract every surface compiles. Same KPI from MEASURE(), from the certified dashboard, and from Genie. If they disagree, the space stays off Slack. Permissions checked as the asker. Inside a Genie Agent, compute is the author’s warehouse and data access is the end user. That split does not automatically hold in Teams, Copilot on maker credentials, or a Databricks App on its default service principal. The row filter still runs. It runs for whoever showed up on the SQL. A written kill line. I use Databricks’ “above 80% before UAT” as the hold line, not the ship line. Ship is 90% and zero fails on a regulated definition. Any miss on that definition takes the space down. A person does that. Not a service principal. Genie Code can draft the Metric View and the MERGE. ZeroOps can propose a fix and wait for Approve. Neither owns the definition. If the job stays green for eleven days because a key was slightly wrong, that is still your incident. I wrote the four pieces up so I would stop re-explaining this in reviews. Argue the 80 vs 90 line if you want. I will not move a regulated KPI to “looks fine in the demo.” submitted by /u/Worth_Phase1358 [link] [comments]

00Worth_Phase13581mo ago
Reddit

Databricks pipeline started failing after moving tables to Unity Catalog — how would you debug it?

Suppose a pipeline was previously using: hive_metastore.schema.table and is migrated to: catalog.schema.table The transformation logic hasn't changed. But the pipeline now fails during execution. Possible areas to investigate: Catalog permissions Schema permissions Service principal Pipeline identity Cluster access mode External location Storage credential Table ownership Hardcoded table references The interesting part is that the SQL may be perfectly valid. How would you troubleshoot this systematically? submitted by /u/hanshu6576 [link] [comments]

00hanshu65761mo ago
Reddit

Databricks pipeline works in one workspace but fails in another — where would you start?

Imagine the same pipeline and code are deployed to two Databricks workspaces. Workspace A: catalog.schema.table → Query works → Pipeline works Workspace B: catalog.schema.table → Catalog not accessible → Pipeline fails The catalog exists, the table exists, and the SQL itself is valid. Would you first investigate: Workspace-to-catalog assignment Unity Catalog permissions USE CATALOG USE SCHEMA SELECT privileges Service principal permissions External locations Storage credentials Cluster access mode SQL Warehouse identity I'm curious about the actual troubleshooting order people follow for these kinds of Unity Catalog issues. What would you check first? submitted by /u/hanshu6576 [link] [comments]

00hanshu65761mo ago
Reddit

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

Every new joiner on our platform team has asked some version of this, so here is the short version. You do not. The Unity Catalog metastore is created in the account console and you have one per region, linked to as many workspaces in that region as you like. Dev, test and prod attach to the same one and share a single catalog.schema.table namespace. The mental model of "each workspace has its own metastore" comes from the legacy Hive metastore, and that is exactly what Unity Catalog replaces. What actually belongs where: - Account: billing, account admins, users, groups, service principals, and the metastore itself. - Workspace: notebooks, clusters, jobs, dashboards, the place people work. - Metastore: catalogs, schemas, tables, views, volumes, and the permissions on them. Identity federation means a group is configured once at account level and assigned to workspaces, so a grant on data applies wherever that group works. Workspace-local groups still exist in non-federated workspaces, but they cannot be granted Unity Catalog permissions, which is a good reason to migrate them. Storage: metastore-level managed storage is optional, and anything already in a bucket becomes an external location with a storage credential. Drew it out as a 2-3 minute animated diagram: https://youtu.be/2yy9-BL_RDo Series in order: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 How are you separating environments inside one regional metastore: catalog per environment, workspace-catalog bindings, or separate regions? submitted by /u/Broad-Cherry-5999 [link] [comments]

00Broad-Cherry-59991mo ago
Reddit

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

This comes up constantly in real platform design, and the confusion is almost always about which layer owns what. The account is the container for the organisation, normally one per cloud provider. Billing, account admins and identities live there. A workspace is a single deployment of the UI: notebooks, clusters, jobs, dashboards. Most teams run dev, test and prod as separate workspaces. The Unity Catalog metastore is the part people place wrongly. It is created at account level, not inside a workspace, and the docs say you have one metastore per region, linked to any number of workspaces in that region. So dev, test and prod in the same region share one metastore and one catalog.schema.table namespace, instead of each keeping a separate legacy Hive metastore. Identities follow the same shape: users, groups and service principals are configured once in the account console and then assigned to workspaces, so you grant a group access to data once rather than repeating it per workspace. Storage is still yours. Metastore-level managed storage is optional, and existing buckets are registered as external locations backed by a storage credential. One line to remember it by: the account holds identities and billing, workspaces hold the tools and compute, the metastore holds the data and who can see it, shared across every workspace in the region. I drew it as a short animated diagram because the hierarchy sticks better visually (2:32): https://youtu.be/2yy9-BL_RDo Episode 3 of a series, in order here: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 For those running dev and prod in the same region: are you isolating with separate catalogs, workspace bindings, or something else? submitted by /u/Broad-Cherry-5999 [link] [comments]

00Broad-Cherry-59991mo ago
Reddit

Can Databricks PAT authentication be used to mint an embed token?

Iam working on a Databricks integration where we support both PAT token and Service Principal authentication. The PAT connection works fine for making Databricks API calls, but when I tried to generate an embed token for dashboard embedding, it doesn’t seem to work. If PAT is not supported, is there any official reason or limitation behind this? Would appreciate any clarification from people who have implemented Databricks dashboard embedding. submitted by /u/Bhanuprakash_1947 [link] [comments]

00Bhanuprakash_19471mo ago
Reddit

What credential should I use to authenticate Databricks Zerobus?

I’m learning how to use Databricks Zerobus and I’m confused about authentication. When connecting to Zerobus, should I use a Databricks personal access token, OAuth credentials, a service principal, or another type of credential? I’m testing this for a learning project and want to follow the recommended secure approach. I won’t post any actual credentials. What authentication method are people using, and are there any setup steps or permissions I should know about? submitted by /u/Stu-dent999 [link] [comments]

00Stu-dent9991mo ago
Databricks CommunityData Engineeringanswered

Managing Service Principal Permissions at Scale: External Locations vs. Managed Volumes

001mo ago
Databricks CommunityData Engineering

System generated Service Principal -App

001mo ago
Reddit

Serverless Access Control is here!

We finally have a simple way to control access to serverless in the workspace. Two built-in objects have been introduced: - Default Interactive Compute - Default Automated Compute The first one supports notebooks and databricks connect. Second one jobs and SDP pipelines. To limit who can use serverless: Click Compute in the workspace sidebar. In the Serverless tab, click the kebab menu next to Default Automated Compute , then click Edit permissions . Remove the All Users group, or the group that includes all workspace users. Add only the specific users, groups, or service principals that you want to authorize. https://preview.redd.it/sg59jole9hkh1.jpg?width=957&format=pjpg&auto=webp&s=0c33d19d04c57856c51f57f33c44719317fe8347 submitted by /u/szymon_dybczak [link] [comments]

00szymon_dybczak2mo ago
Databricks CommunityData Engineering

Declarative Asset Bundle Service Principals best practices

003mo ago
Databricks CommunityAdministration & Architectureanswered

Zerobus ingestion fails in Databricks Free Edition using Service Principal OAuth

004mo ago
RedditDiscussion

Why Databricks Genie returns MessageStatus.FAILED in App API (but works fine in UI) — solved

Hey fellow Databricks builders, Our team at **Enqurious** just wrapped a hackathon building a full medallion architecture (Bronze → Silver → Gold) with Genie AI integration for natural language querying. We hit a wall that cost us 6+ hours — sharing the full fix so you don't have to. # What was the setup? * Gold layer tables in Unity Catalog * Genie Space configured and working in UI * Databricks App deployed successfully # What causes MessageStatus.FAILED in Databricks Genie App API? The root cause is a missing service principal permission chain. The Databricks App runs under a **service principal** — not your user account. Genie working in the UI only proves *your user* has access, not the service principal the App uses at runtime. This is why the same question works in the Genie UI and fails via the App API every time. # What is the complete permission fix? The service principal needs **all five** of these — missing any one causes silent failures: sql -- 1. Catalog level (most commonly missed) GRANT USE CATALOG ON CATALOG your_catalog TO `your-app-service-principal`; -- 2. Schema level GRANT USE SCHEMA ON SCHEMA your_catalog.your_schema TO `your-app-service-principal`; -- 3. Table level — each table individually GRANT SELECT ON TABLE your_catalog.your_schema.table_1 TO `your-app-service-principal`; GRANT SELECT ON TABLE your_catalog.your_schema.table_2 TO `your-app-service-principal`; **Plus in the UI:** * SQL Warehouses → your warehouse → Permissions → Add service principal → **Can Use** * Genie Space → Share → Add service principal → **Can Run** # Where do you see the real error (not just MessageStatus.FAILED)? The actual error is in the **Genie Monitoring tab** inside your Genie Space — not the App logs. App logs only show the generic `MessageStatus.FAILED`. The Monitoring tab shows the specific SQL error or permission denial that caused it. This single tip would have saved us 3 of those 6 hours. # How long do Databricks permissions take to propagate? Unity Catalog permissions take **5–15 minutes** to propagate. Testing immediately after granting permissions is the most common reason the fix appears not to work. Wait 10 minutes, hard-refresh your browser, then retest. # Quick debug checklist If you're still hitting `MessageStatus.FAILED`: □ Check Genie Monitoring tab for the actual SQL error □ Confirm service principal has USE CATALOG (not just SELECT) □ Confirm SELECT on each table individually — schema-level alone is not enough □ Confirm SQL Warehouse is Running, not Stopped □ Wait 10 minutes after any permission change before retesting □ Hard refresh browser before retesting □ Verify app.yaml uses ["streamlit", "run", "app.py"] — not ["python", "app.py"] # FAQ **Q: Why does Genie work in the UI but fail in the Databricks App?** A: The App runs under a service principal, not your user account. The service principal needs its own separate permission chain regardless of your personal access. **Q: Is USE CATALOG required even if SELECT is granted on all tables?** A: Yes — USE CATALOG is mandatory. Without it the service principal cannot traverse the catalog hierarchy, even with table-level SELECT grants. **Q: Where is the real error when Genie fails in an App?** A: Genie Space → Monitoring tab. App logs only show the generic failure; Monitoring shows the specific cause. **Q: How long do Unity Catalog permissions take to propagate?** A: 5–15 minutes. Always wait before retesting. The Enqurious team wrote up a full walkthrough with SQL commands, screenshots, and the complete [app.py](http://app.py) template Happy to share our Genie Space config or [app.py](http://app.py) in the comments — just ask. **Questions for the community:** * Has anyone else faced similar Genie + App permission issues? * Is there a better way to debug these permission chains? * Would love to hear if Databricks is working on si […truncated]

53Square-Mix-13025mo ago
RedditTutorial

Read-only SQL wrapper for using LLM agents safely on Databricks (open source)

I've been running LLM agents against my Databricks workspace daily across Claude Code, Codex, and Cursor IDE. Hit two issues: 1. The CLI happily runs whatever SQL you pass. Nothing stops a confused agent from a `DROP TABLE`. 2. The default JSON output bloats the agent's context: `[{"col_0":"1"}]` when you just want one number back. So I built a small wrapper around `databricks experimental aitools tools query` and open-sourced it as a drop-in agent skill in my `databricks-bundle-template` repo. **What it does:** - Allow-lists six read-only prefixes: SELECT, WITH, SHOW, DESCRIBE, DESC, EXPLAIN - Block-lists every destructive verb (INSERT, UPDATE, DELETE, MERGE, DROP, TRUNCATE, GRANT, COPY INTO, and the rest) - Strips comments and quoted strings before validation, so `SELECT '/* DROP TABLE foo */ 1'` doesn't slip past - Shapes output to row geometry: 1 cell becomes a scalar, 1 column becomes lines, NxM becomes TSV. The agent reads `1` instead of `[{"col_0":"1"}]`. The wins compound on larger queries. **Install:** ``` databricks bundle init https://github.com/vmariiechko/databricks-bundle-template --template-dir assets/dbx-ro-query ``` Single file Python, no dependencies. Works with Claude Code, Codex, Cursor IDE, and anything that follows the agentskills.io layout (Gemini CLI, OpenCode, Antigravity, etc. pick up the `.agents/skills/<name>/SKILL.md` convention). **Caveats:** - Upstream `aitools tools query` is in the experimental namespace - Verb-based block-list, so `SELECT 1 AS drop` is rejected (rename the alias) - Not a substitute for warehouse RBAC. It's a defense-in-depth layer; still run agents under a service principal with read-only permissions Repo: https://github.com/vmariiechko/databricks-bundle-template/tree/main/assets/dbx-ro-query Curious what others are using for safe LLM agent access to Databricks data.

45Marik3485mo ago
RedditHelp

Databricks Streaming Table + ABAC policies causing ABAC_POLICIES_NOT_SUPPORTED

Guys... has anyone run into this before? We’re facing an issue in Databricks with a streaming pipeline writing data into a Streaming Table that has ABAC policies applied on some columns (column masking policies). When the pipeline tries to write/update the table, it fails with this error: `ABAC_POLICIES_NOT_SUPPORTED: ABAC policies are not supported on tables defined within a pipeline.` Basically, the service principal running the job cannot update the Streaming Table because the table has ABAC policies applied. What’s confusing is that the Databricks docs mention that Streaming Tables support ABAC. We also tried adding the service principal to the policy whitelist/exemptions, but it still fails. Has anyone seen this behavior before? Is this an actual limitation with Lakeflow/Streaming Tables + ABAC, or are we missing some configuration? Thanks!

53Quaiada5mo ago
RedditNews

What's new in AIBI Dashboards - April 2026

* **Publish with service principal credentials**: Authors can publish dashboards using the data credentials of a service principal. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/share/share#publish-dashboard) * **Service principal ownership**: Workspace admins can transfer dashboard ownership to a service principal in the UI. 📖 [Documentation](https://docs.databricks.com/aws/en/ai-bi/admin/#transfer-ownership) * **Choropleth map admin levels**: Choropleth maps support US admin levels 3 (regions, multi-state groupings) and 4 (states). 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/visualizations/maps) * **SQL editor line numbers**: The SQL query editor displays line numbers to help with legibility and debugging. * **PDF subscription page selection**: Dashboard authors can select which pages to include in PDF email subscriptions. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/share/schedule-subscribe) * **Parameter values in widget titles and descriptions**: Dashboard authors can reference parameter values in widget titles and descriptions, so the text updates dynamically as viewers change parameter selections. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/filters/parameters) * **Table cross-filtering and drill-through**: Tables support cross-filtering and drill-through. * **Counter prefix and suffix**: Numbers in counters support custom prefixes and suffixes. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/visualizations/types#counter) * **Schema browser default dataset type:** Adding a table to a dashboard from the schema browser creates a [local metric view](https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/local-metric-views) by default instead of a SQL dataset. * **Warehouse overload message**: Dashboards show a message explaining when rendering is delayed due to the warehouse being overloaded. * **Tabular attachments in email subscriptions**: Dashboard email subscriptions include tabular attachments. * **Fullscreen scroll position**: Exiting fullscreen mode on a published dashboard returns you to your previous scroll position instead of jumping to the top of the page. * **Local metric views**: A new dataset type lets you create metric views directly in a dashboard using a low-code visual interface, without publishing to Unity Catalog first. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/local-metric-views) * **Edit hex color values inline**: Authors can click directly on a hex color value to edit it in place. * **View SQL for visualization widgets**: Authors can view the SQL behind specific visualization widgets while in draft mode. * **Waterfall chart totals**: Waterfall charts with categorical X-axis support a total bar. * **Scatter plot shape field**: Scatter plots support a shape field to differentiate data points by category. * **Clear applied filters individually**: Dashboard viewers can individually clear applied filters from the active selection bar. * **Text box vertical alignment**: Text box widgets support vertical alignment (top, center, and bottom). * **Choropleth map boundaries**: Choropleth maps support additional boundary types, including ZIP code and NUTS regions. * **“Explain this change” chart types**: The “Explain this change” feature is available for pivot table cells, horizontal bar charts, pie charts, and heatmaps, in addition to time series charts. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/genie-spaces#explain-chart-changes)

155Youssef_Mrini5mo ago