Skip to content
LakeFlowChangelog · September 2026

Lakeflow Connect added managed marketing connectors and Lakeflow Jobs released task output in for_each loops.

Databricks expanded Lakeflow Connect with native managed connectors for ad and marketing platforms, including Salesforce, HubSpot, Google Ads, and Meta Ads, landing raw ingestion data into Unity Catalog [29]. Accompanying guides demonstrated how to model these feeds for marketing analytics [28]. Within Lakeflow Jobs, support for passing task outputs inside for_each task loops entered public beta [3], while Databricks delivered updates to Lakeflow Designer [14].

Teams adopting Lakeflow Connect shared early production experiences and raised infrastructure questions [19]. Users deploying the SQL Server change data capture gateway asked how to select specific virtual machine types [10], handle elastic compute requirements [1], and avoid connection saturation on source transactional databases [22]. Practitioners also shared architectural patterns for managing upstream schema drift and executing historical backfills alongside ongoing change streams [6].

Lakeflow Jobs sparked discussion around production orchestration maturity. Teams evaluated replacing legacy Airflow schedules with data-aware triggers in Lakeflow [16, 26], though engineers debated whether the platform provided adequate tooling for service level objectives and SLA monitoring across large fleets [15, 25]. Community members also worked through practical job configurations, such as implementing conditional task retries [8] and enabling predictive optimization for streaming tables in declarative pipelines [18].

Everything cited

  1. [1]Lakeflow connect SQL server ingestion can be made elastic? community · 2026-09-30
  2. [2]system.query.history is now Generally Available in Databricks community · 2026-09-30
  3. [3]Lakeflow Jobs update! Task output in foreach now available in Beta! community · 2026-09-30
  4. [4]Time to Swap the Cookies for Jetfuel - New Dataset & New Databricks Genie Tutorial community · 2026-09-29
  5. [5]Trigger a job automatically on databricks DABs bundle deploy video · 2026-09-28
  6. [6]Operationalizing Lakeflow Connect:Handling Upstream Schema Evolution & Historical Backfill community · 2026-09-24
  7. [7]Running open-Jev in SQL on Databricks news · 2026-09-24
  8. [8]Conditionally retrying a task on Lakeflow Jobs community · 2026-09-24
  9. [9]How I built agent-based security reviews on Databricks news · 2026-09-24
  10. [10]How can I configure Lakeflow Connect SQL Server CDC Gateway to use a desired VM type? community · 2026-09-24
  11. [11]I didn't want to choose between doing engineering and staying up to date with Databricks, so I built this. Hope it's useful to you too! community · 2026-09-23
  12. [12]Real-Time SAP Accounts Receivable on Databricks Lakeflow Declarative Pipelines community · 2026-09-23
  13. [13]Data Engineering Project On Real Company | Real Data | Pyspark | Databricks | Production Ready community · 2026-09-23
  14. [14]New features in Lakeflow Designer community · 2026-09-22
  15. [15]Anyone running actual SLOs on Lakeflow Jobs, or is it still “it failed, rerun it”? community · 2026-09-22
  16. [16]Do you still use Airflow with Databricks? community · 2026-09-21
  17. [17]Environments on classic compute community · 2026-09-19
  18. [18]Predictive Optimization for Streaming tables in Lakeflow pipelines community · 2026-09-19
  19. [19]Lakeflow Connect in production: What has your experience been? community · 2026-09-19
  20. [20]system tables data retention community · 2026-09-18
  21. [21]BrickTalk Recording | One Platform, Any Source: Unifying Enterprise Data with Lakeflow Connect community · 2026-09-18
  22. [22]Lakeflow connect CT pipeline keeps running multiple connections in source community · 2026-09-15
  23. [23][Discussion] Lakeflow Jobs: If your data catalog could trigger a job on any change what would it be? community · 2026-09-15
  24. [24]Private Network Gateway community · 2026-09-13
  25. [25]Best practices for SLA monitoring and automated retries across hundreds of Lakeflow Jobs community · 2026-09-12
  26. [26]igrating from Cron-based Airflow to Lakeflow's Data-Aware Triggers — Real-world experiences? community · 2026-09-12
  27. [27]Lakeflow Connect SQL Server Connector community · 2026-09-12
  28. [28]Tutorial: Transform your Lakeflow Connect ad data into visual and conversational analytics community · 2026-09-11
  29. [29]Unify your marketing data with Lakeflow Connect news · 2026-09-11
  30. [30]What is Data Transformation? news · 2026-09-03