Skip to content

Road to a data engineer

Google AI Platform

Road to a data engineer

Current situation and learning path
Data Engineering Landscape
Databases
Databases
- Data warehouse toolkit notes
- Database Design video course and notes
Decision Journal
Decision Journal
- Decision: Choice of cloud platform to learn
- Decision Journal Template
Google Cloud Platform
Google Cloud Platform
- Common GCP code snippets
- Data Engineering Certificate
  Data Engineering Certificate
- Qwiklabs
  Qwiklabs
Jobs and Roles
Jobs and Roles
Linux
Linux
- Data Analytics at the terminal
  Data Analytics at the terminal
  - Crash Course
    Crash Course
    
    Introduction and Motivation
    
    Fundamental Bash Commands
    
    Expanding the fundamentals for Data Analysis
    
    Downloading files with Bash
    
    Working with and manipulating text files
    
    BONUS - Common text file cleaning operations
    
    Setup data tools
    
    csvkit - a brief introduction
    
    08 getting data into sqlite
    
    Interacting with SQLite databases
    
    Common data cleaning and transforming tasks in SQL
    
    Replicating common spreadsheet tasks in SQL
    
    Building a pipeline as a bash script
    
    Additional resources
  - Deepnote Notebooks
    Deepnote Notebooks
    
    Environment setup
    
    Exploring terminal functionality for interacting with data
    
    GREP basics
    
    Getting a feel for the data
    
    Saving data to SQLite at the terminal
    
    CSV and SQLite tools overview
Projects
Projects
- Bitcoin Price Dashboard
  Bitcoin Price Dashboard
- fivethirtyeight data to BigQuery
  fivethirtyeight data to BigQuery
Python
Python
- Apache Airflow
- Data Engineering with python
  Data Engineering with python
- Google Colab
  Google Colab
  - Converting between video types
  - Backup of Domo Cards
  - Intro to Google Colab Notebooks
    Intro to Google Colab Notebooks
    
    What is Google Colab
    
    Using Google Drive as a filesystem in Google Colab
    
    Reading from and saving to Google Sheets
    
    Accessing Adobe Analytics data through the API
    
    Query BigQuery from within Google Colab
- Pycon US 2021
  Pycon US 2021
Resources
Resources

Google AI Platform

AI Platform can be used to train and host machine learning models.

AI Platform overview

Made with Material for MkDocs