Databricks Data Engineering with AI Build a Lakehouse

Master medallion architecture on Databricks: PySpark, SQL, Unity Catalog, Genie Agent and Testing

What you'll learn

⚡ Build a complete medallion lakehouse on Databricks (Bronze, Silver, Gold) from an empty workspace, using real New York City taxi data
⚡ Use Genie Code to generate first-draft PySpark and SQL, then refactor it into typed, tested, production-grade modules
⚡ Write unit tests with pytest, and verify each stage against a known row-count anchor using an independent SQL query
⚡ Set up Unity Catalog, schemas and Volumes, and version the whole project in a Git-backed repository
⚡ Govern your lakehouse with Unity Catalog: catalogs, schemas, permissions and lineage across every layer
⚡ Build a semantic layer with Unity Catalog Metric Views: define each business metric once, then query it across any dimension
⚡ Query Metric Views with the MEASURE() clause, so every dashboard and question reads from one governed definition.
⚡ Set up a Genie Space so business users can ask questions of your Gold data in plain English.
⚡ Configure Genie with instructions, example SQL and trusted assets, and learn its limits and best practice.
⚡ Build AI/BI Dashboards on Databricks: visualise KPIs, add filters, and publish them for stakeholders.
⚡ Commit your work to GitHub at every step, building a clean audit trail.
⚡ Understand how Spark works: the DataFrame, lazy evaluation, the driver and executors, and Delta Lake.

Requirements

❗ Basic Python. You can read and write simple functions.
❗ Basic SQL. You can write SELECT queries.
❗ A Databricks account. A free option is enough, and the early lessons walk you through the setup.
❗ A GitHub account. It is free.
❗ No prior Databricks, Spark or data engineering experience is required.

Description

AI can write data engineering code in seconds. Turning that first draft into a production-grade pipeline is the real skill, and it is what this course teaches.
You will build a complete lakehouse on Databricks, processing millions of real New York City taxi trips through a medallion architecture: raw Bronze, cleaned Silver, aggregated Gold. You will use Genie Code, Databricks' built-in AI assistant, to generate first drafts, then refactor them into typed, tested, version-controlled modules that meet a clear set of coding standards. This is a hands-on project, not a tour of features. You start with an empty workspace and finish with a full solution you can show an employer.
The course goes past the pipeline. You build it, you govern it, and you serve it to real users. Along the way you will
✨ Set up Unity Catalog, schemas, Volumes, and a Git-backed repository from scratch
✨ Ingest raw data into a Bronze Delta table and stamp lineage onto every row
✨ Clean and conform the data in Silver, then aggregate it into Gold tables
✨ Write and refactor PySpark and SQL across all three layers
✨ Write unit tests with pytest, build reusable fixtures, and run them inside Databricks
✨ Verify every stage against a known row-count anchor, on a separate engine, so your numbers are always right
✨ Commit your work to GitHub at each step, building a clean audit trail
Then you make the data useful. You govern the lakehouse with Unity Catalog, so access and lineage stay under control. You build a semantic layer with Unity Catalog Metric Views, where each business metric is defined once and queried across any dimension with the MEASURE() clause. You set up a Genie Space, so business users can ask questions of your Gold data in plain English, backed by trusted metrics. And you build AI/BI Dashboards to track the KPIs and share them with stakeholders.
You will also learn the ideas behind the code, so nothing is a black box. The course explains how a Spark DataFrame works, why Spark is lazy, how the driver and executors split the work, and what Delta Lake gives you. You will review AI-generated code against eight coding standards, and configure the assistant with those standards so its output lands closer to production-ready every time.
The rhythm is simple and repeatable: build it, verify it, commit it. By the end you will have a portfolio-ready, end-to-end lakehouse, from raw files to governed metrics and dashboards, with AI as your assistant rather than your autopilot.
This course suits data engineers, analysts, and BI developers who know some Python and SQL and want to work the modern, AI-assisted way on Databricks. No prior Databricks experience is required.

Who this course is for

⭐ Data engineers who want a modern, AI-assisted workflow on Databricks
⭐ Data analysts who know some SQL and want to move into data engineering.
⭐ Python and software developers moving into data work.
⭐ Anyone who wants a real, portfolio-ready lakehouse project, not a feature tour.
⭐ People who want AI to write code faster, and want to make that code production-grade.

Homepage

https://www.udemy.com/course/databricks-data-engineering-with-ai-build-a-lakehouse

Rapidgator
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part1.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part2.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part3.rar.html
DDownload
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part1.rar
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part2.rar
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part3.rar

KatFile
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part1.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part2.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part3.rar.html
FreeDL
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part1.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part2.rar.html
yhpco.Databricks.Data.Engineering.with.AI.Build.a.Lakehouse.part3.rar.html