Decision: Choice of cloud platform to learn
1st March 2021
Which cloud service to learn to pivot into a data engineering role.
Mental / Physical State
- [ ] Energised
- [ ] Focussed
- [x] Relaxed
- [ ] Confident
- [ ] Tired
- [ ] Accepting
- [ ] Accomodating
- [ ] Anxious
- [ ] Resigned
- [ ] Frustrated
- [ ] Angry
The Situation / context
Modern data engineers use cloud platforms to perform tasks required.
Familiarity with at least 1 cloud platform is required to enter a data engineering role.
Notable Cloud platforms include:
- Amazon Web Services
- Azure (Microsoft)
- Google Cloud Platform
I am already familiar with various aspects of Google Cloud platform.
The problem statement or frame
There are multiple cloud platforms that could be employed for data engineering.
Becoming familiar with all of them is not the best use of time.
It is necessary to choose a single Cloud platform to become proficcient in.
Learn how to perform key data engineering tasks using Google Cloud Platform.
The Variables that govern the situation include
AWS has the largest market share Globally and within Australia.
Learning AWS will provide the greatest optionality for future opportunities.
I have some experience with Google Cloud Platform.
Woolworths (current employer) have made a company wide decision that the cloud platform of choice will be Google Cloud Platform.
If I want to transition to data engineer within Woolworths I will need to be competent with GCP.
Woolworths will provide training materials to become proficient on GCP.
Alternatives that were seriously considered and not chosen were
AWS and Azure
Explain the range of outcomes
Best case transition into a data engineering role within current employer
- Transition into a data engineering role outside current employer with GCP skills
- Fail to transition into a data engineering role within current employer with GCP skills
- Fail to transition to data engineering role outside of current employer with GCP skills
What I expect to happen and the actual probabilities are
Best case is medium to high probability (75%) given the following are met:
- GCP data engineer certification
- Demonstrate capability with Cloud composer, data flow, Google Cloud functions, Big Query.