Skip to content
Road to a data engineer
Scheduling and Monitoring
Road to a data engineer
Current situation and learning path
Data Engineering Landscape
Databases
Databases
Data warehouse toolkit notes
Database Design video course and notes
Decision Journal
Decision Journal
Decision: Choice of cloud platform to learn
Decision Journal Template
Google Cloud Platform
Google Cloud Platform
Common GCP code snippets
Data Engineering Certificate
Data Engineering Certificate
Data Engineering Certificate Misc Notes
Data Engineering practice Exam notes
Google AI Platform
BigQuery
Google Cloud Bigtable
Cloud SQL
Google Dataflow
Google Datalab
Google Dataproc
Google Datastore and Firestore
Google Memorystore
Google Pub/Sub
Google Cloud Spanner
Google Storage
Google Tensorflow
Qwiklabs
Qwiklabs
Badges earned on Qwiklabs
Baseline Data ML AI Quest
Baseline Infrastructure Quest
BigQuery for data analysts quest
BigQuery for Marketing Analysts
BigQuery for Data Analysis Quest
BigQuery for Data Warehousing Quest
Data Engineering Quest
Integrate Machine Learning APIs Quest
Jobs and Roles
Jobs and Roles
Conversation with chapter lead data engineer
Junior data engineer at woolies01
Interview prep
Interview prep
Data Engineering Interview Prep
Flashcard template
BigO Notation
Bubble sort
Flashcard 20231215
Flashcard 20231216
Flashcard template
Searching and sorting algorithms in python
Recursion
Hashtables
Bloom filters
Graphing Algorithms
Python Codewars problems
Leetcode problems solved with Python
Linux
Linux
Data Analytics at the terminal
Data Analytics at the terminal
Crash Course
Crash Course
Introduction and Motivation
Fundamental Bash Commands
Expanding the fundamentals for Data Analysis
Downloading files with Bash
Working with and manipulating text files
BONUS - Common text file cleaning operations
Setup data tools
csvkit - a brief introduction
08 getting data into sqlite
Interacting with SQLite databases
Common data cleaning and transforming tasks in SQL
Replicating common spreadsheet tasks in SQL
Building a pipeline as a bash script
Additional resources
Deepnote Notebooks
Deepnote Notebooks
Environment setup
Exploring terminal functionality for interacting with data
GREP basics
Getting a feel for the data
Saving data to SQLite at the terminal
CSV and SQLite tools overview
Projects
Projects
Bitcoin Price Dashboard
Bitcoin Price Dashboard
Bitcoin Dashboard Project Details
Bitcoin dashboard Cloud Function code
Scheduling and Monitoring
Scheduling and Monitoring
Table of contents
Cloud Scheduler details
Monitoring with Healthchecks.io
Bitcoin Data Transformation
Bitcoin project visualisation
fivethirtyeight data to BigQuery
fivethirtyeight data to BigQuery
Project details
Airflow server setup
Airflow screenshots
fivethirtyeight DAG
Project visualisations
Python
Python
Apache Airflow
Data Engineering with python
Data Engineering with python
Chapter 1: What is data engineering
Chapter 2 - Notebook
Chapter 2: Building Our Data Engineering Infrastructure
Chapter 3 Notebook
Chapter 3: Reading and writing files
Chapter 4 - Notebook
Chapter 4: Working with Databases
Misc
Google Colab
Google Colab
Converting between video types
Backup of Domo Cards
Intro to Google Colab Notebooks
Intro to Google Colab Notebooks
What is Google Colab
Using Google Drive as a filesystem in Google Colab
Reading from and saving to Google Sheets
Accessing Adobe Analytics data through the API
Query BigQuery from within Google Colab
Pycon US 2021
Pycon US 2021
Keynote: Robert Erdmann
Creating extensible workflows
What we learned from Papermill to operationalize notebooks
Writing Good Documentation for Developers
Google: Serverless Application Architecture
Build Slack apps fast in Python
Resources
Resources
Code snippets
Data Engineering through osmosis
Links
Project ideas
Videos
Table of contents
Cloud Scheduler details
Monitoring with Healthchecks.io
Scheduling and Monitoring
Cloud Scheduler details
Monitoring with Healthchecks.io
