IBM - ETL and Data Pipelines with Shell, Airflow and Kafka
5.0 /5- Offered byCoursera
- Public/Government Institute
ETL and Data Pipelines with Shell, Airflow and Kafka at Coursera Overview
Duration | 11 hours |
Total fee | Free |
Mode of learning | Online |
Difficulty level | Intermediate |
Official Website | Explore Free Course  |
Credential | Certificate |
- Overview
- Highlights
- Course Details
- Curriculum
- Student Reviews
ETL and Data Pipelines with Shell, Airflow and Kafka at Coursera Highlights
- Flexible deadlines Reset deadlines in accordance to your schedule.
- Shareable Certificate Earn a Certificate upon completion
- 100% online Start instantly and learn at your own schedule.
- Course 11 of 13 in the IBM Data Engineering
ETL and Data Pipelines with Shell, Airflow and Kafka at Coursera Course details
- After taking this course, you will be able to describe two different approaches to converting raw data into analytics-ready data. One approach is the Extract, Transform, Load (ETL) process. The other contrasting approach is the Extract, Load, and Transform (ELT) process. ETL processes apply to data warehouses and data marts. ELT processes apply to data lakes, where the data is transformed on demand by the requesting/calling application.
- Both ETL and ELT extract data from source systems, move the data through the data pipeline, and store the data in destination systems. During this course, you will experience how ELT and ETL processing differ and identify use cases for both.
- You will identify methods and tools used for extracting the data, merging extracted data either logically or physically, and for importing data into data repositories. You will also define transformations to apply to source data to make the data credible, contextual, and accessible to data users. You will be able to outline some of the multiple methods for loading data into the destination system, verifying data quality, monitoring load failures, and the use of recovery mechanisms in case of failure.
ETL and Data Pipelines with Shell, Airflow and Kafka at Coursera Curriculum
Data Processing Techniques
ETL Fundamentals
ELT Basics
Comparing ETL to ELT
Data Extraction Techniques
Introduction to Data Transformation Techniques
Data Loading Techniques
Course Introduction
Summary & Highlights
Practice Quiz: ETL and ELT Processes
Graded Quiz: ETL and ELT Processes
ETL & Data Pipelines: Tools and Techniques
ETL using Shell Scripting
Introduction to Data Pipelines
Key Data Pipeline Processes
Batch Versus Streaming Data Pipeline Use Cases
Data Pipeline Tools and Technologies
Linux Commands and Shell Scripting
ETL Techniques
Summary & Highlights
Summary & Highlights
Practice Quiz: ETL using Shell Scripts
Graded Quiz: ETL using Shell Scripts
Practice Quiz: An Introduction to Data Pipelines
Graded Quiz: An Introduction to Data Pipelines
Building Data Pipelines using Airflow
Apache Airflow Overview
Advantages of Using Data Pipelines as DAGs in Apache Airflow
Apache Airflow UI
Build DAG Using Airflow
Airflow Monitoring and Logging
Summary & Highlights
Practice Quiz: Using Apache Airflow to build Data Pipelines
Graded Quiz: Using Apache Airflow to build Data Pipelines
Building Streaming Pipelines using Kafka
Distributed Event Streaming Platform Components
Apache Kafka Overview
Building Event Streaming Pipelines using Kafka
Kafka Streaming Process
Summary & Highlights
Practice Quiz: Using Apache Kafka to build Pipelines for Streaming Data
Graded Quiz: Using Apache Kafka to build Pipelines for Streaming Data
Final Assignment
Project Overview
Congrats & Next Steps
Team & Acknowledgements
Final Quiz
Other courses offered by Coursera
ETL and Data Pipelines with Shell, Airflow and Kafka at Coursera Students Ratings & Reviews
1 Rating- 4-51
Student Forum
Anything you would want to ask experts?