Current situation and learning path

Goal

Transition into a data engineering role.

Data Engineering

What is it?

What skills are needed?

https://awesomedataengineering.com/

• SQL
• Python
• At least 1 cloud computing platform
• Relational Databases
• 1 pipeline management tool / platform

Current competencies

SQL - intermediate

Python for data analysis - Pandas.

Working knowledge of GCP tools: - BigQuery - Cloud Functions - App Engine - DataPrep - Google Cloud Storage

skill gaps identified

Advanced SQL skills. specifically around creating and modifying data. Stored procedures.

GCP - data engineering certificate.

Python Airflow

Data warehousing principles.

Roadmap

1. Google Cloud platform certification

• On tools training through qwiklabs
• Review of GCP documentation
• Review of exam materials
• exam course guide
• pass data engineering certification exam

2. Level up SQL

3. Python Specific for data engineering

Work through "Data Engineering with Python" book.

4. Data warehousing

Work through Kimball data warehousing book.

5. Projects

• Create serverless web analytics functionality
• Create cryptocurrency dashboard
• Review commercial ETL tools such as: Stitch data, Panoply, FiveTran, Airbyte