Advanced Data Engineering with Databricks
- Offered byDatabricks
Advanced Data Engineering with Databricks at Databricks Overview
Duration | 16 hours |
Total fee | ₹1.27 Lakh |
Mode of learning | Online |
Difficulty level | Advanced |
Official Website | Go to Website  |
Credential | Certificate |
Advanced Data Engineering with Databricks at Databricks Highlights
- Earn a certificate after completion of course from Databricks
Advanced Data Engineering with Databricks at Databricks Course details
The "Advanced Data Engineering" course is designed for professionals who are looking to deepen their expertise in the field of data engineering and tackle sophisticated data challenges
In this course, students will build upon their existing knowledge of Apache Spark, Structured Streaming, and Delta Lake to unlock the full potential of the data lakehouse by utilizing the suite of tools provided by Databricks
This course places a heavy emphasis on designs favoring incremental data processing, enabling systems optimized to continuously ingest and analyze ever-growing data
Advanced Data Engineering with Databricks at Databricks Curriculum
Incremental Processing with Spark Structured Streaming and Delta Lake
Streaming Data Concepts
Introduction to Structured Streaming
Aggregations, Time Windows, Watermarks
Delta Live Tables Review
Auto Loader
Streaming ETL Patterns with DLT
Data Ingestion Patterns
Data Quality Enforcement Patterns
Data Modeling
Streaming Joins and Statefulness
Data Privacy Patterns
Store Data Securely
Streaming Data and CDF
Deleting Data in Databricks
Performance Optimization with Spark and Delta Lake
Spark Architecture
Designing the Foundation
Introduction of Spark UI
Fine-Tuning - Choosing the Right Cluster
Code Optimization
Shuffles
Spill
Skew
Serialization
SWE Practices for Delta Live Tables Pipelines
Automate Production Workflows
Introduction to REST API and CLI
Deploy Batch and Streaming Jobs
Working with Terraform
Other courses offered by Databricks
Student Forum
Anything you would want to ask experts?