Going from Analytics to Data Engineering
I've been in analytics (Data Analyst, BI Analyst) for 7 years now, and I want to move to Data Engineering. I'm very comfortable with SQL/Tableau, have working knowledge in Python (write little automation and ETL scripts with Pandas etc), have done a ML thesis in grad school, and have access to all AWS tools. How should I start making moves?
I think you need to be able to build CDK pipelines. ML isn't too relevant for a DE. A DE is a specialist SDE who focuses on data and data hardware.
I’m in a very similar situation - in fact I actually just got an offer as a data engineer and will be starting in a few weeks. One difference with my path, though, is that I went from Analytics/BI after 8 years to Software Development for a few years. However, most items I spoke about in interviews related to building data warehouses while in an analytics role. One problem I’ve noticed with data engineering is that companies will slap that term on a position and some of the roles are more like “real-time/streaming data to multiple services data engineer” and others are more like “we need to organize some tables we have and make some visualizations”. I flunked out of interviews fast that asked about streaming stuff. So basically what I’m saying is that it might be more worth your time to focus on the more “analytics” oriented roles, or even look at an Analytics Engineer position.
Congratulations!! I am interested in Analytical Engineering roles and have similar BI/Analytical background as OP. Which companies did you target ? I have largely seen Analytical Eng roles at Netflix
My first pass was to target any company that had a position that said "Analytics Engineer" in the title. Later I targeted Data Engineering roles since I'm not sure I wanted to stay in the analytics space. So there's not really a specific company I was targeting. Initially it was Netflix but then other places looked good too.
Learn Java and Scala
This is very company specific (with regard to Java). Python is the more data engineering focused language. Java will give you more options outside of data engineering (although I think it’s a clunky\ underperforming pos that is the next COBOL).
I'm thrown off by the java recommendations, I was under the impression that python was the priority as well. But maybe they know something I don't.
It’s not really that hard to transition within data. Just simply apply to any DE roles you find and alter your resume to include more tools/skills and work you’ve done that cater towards data engineering. I’m a Data Scientist and was able to get a Data Engineer offer even though I don’t have any experience as a Data Engineer
Can you tell me a bit more? Like what tools you have learned to land the job. I am a DS and am thinking about to be a DE later
Hi, can you please tell what made you move from DS to DE role?
Correlation one has a great program focused on Data Engineering. You'll learn some new concepts, have a chance to work on a capstone project, have the opportunity to get a mentor, and some networking opportunities. Teaching staff for their programs have been top notch. https://www.correlation-one.com/ds4a-data-engineering
Thanks
I am also in the same situation. Trying to change from being a data analyst to Data engineer.. Upskilling my cloud skills as of now..
I'm hiring for a role that might be a good fit for your background and allow you to slowly transition to Data Engineering. DM me if interested.
Can I dm you I have know on airflow too 😀
Can I DM you as well?
Tc?
Did you work on any etl orchestration tool either for batch or real time or it’s just a python code to transform data ?
I have a few years experience in Alteryx. No Airflow or anything.