Member-only story
Data Engineering Part 1 — How to become a Big Data Engineer
How I self taught myself Data Engineering via Hive/Spark-Sql ETL project on Amazon EMR, with project & code.
Hey Readers, I am a Data Science Student and recently I have started learning more about Data Engineering. Data Science and Data Engineering teams co-exist in all data organizations. The Data Scientists are responsible for getting insights from data and applying machine learning on data, while the Data Engineers help the Data Scientist get access to the data and make their models run over huge datasets.
Though a Data Scientist is not expected to master Data Engineering but some small efforts in understanding the concepts can help us in great ways.
In this post, I will explain how I self taught myself Data Engineering in last few weeks.
Disclaimer: Code and Concepts ahead !
If you are just getting started with Data Engineering, it might be worth checking my blogs on SQL basics and getting started with Spark and Zeppelin.
