I was a php developer until year and half ago and then got an opportunity to move to Data Engineering writhing the organization. In this Year and half I have worked on Airflow, GCP, Pyspark and some Java based micro service I have been leading the team for last 2 years which includes running scrum , technical roadmap, connecting with product team to brainstorm new ideas and have experience with managing 3-4 software engineers as well. Although I was new to Data Engineering my overall 9 years of overall experience served me well that within 6 months I started leading data Engineers and my manager trusted me with that. I have started looking out for new opportunities in Data Engineering field but I don’t have much exposure on Hadoop, ok knowledge of sql , okish java. I am confused regarding what kind of roles should I apply for? Target FAANGMULA Data Engineer or Senior Software Engineer or Engineering Manager. My ultimate goal is to get to management but with strong tech skills TC: 225k Yoe : 9 Target TC: 500k+#engineering #software #swe
Hey I’m a php engineer as well, does wayfair have a decent php stack?
Good enough that I wanted to move away from it.
Happy for you fam, I am still trying to escape php even though I work with strictly backend laravel. Good luck with your search!
World Conflicts
8h
497
Understanding Fear: “Allah O Akbar” vs. “May God Bless You” in Public Settings
Tech Industry
Yesterday
2178
15 years age difference, need advice! 🙏
India
Yesterday
1072
Why Worshipping Lord Ram Important in Hinduism?
Tech Industry
1h
1169
Sick of South Indians
Personal Finance
8h
1642
Retire by 45? Rate my NW, lifestyle creep and portfolio
Aside from PHP and data engineering, how is the rest of your framework/stack knowledge? If you’re going all-in on data engineering, round off your skills with a heavier SQL and data warehouse workload, preferably with Azure/AWS/Snowflake data warehouses. Seems like you have good exposure to distributed file-store processing and I’m assuming data lake store, but the data world is turning back around to the relational store in data warehouses, ELT, and SQL (data warehouses have gotten much, much better the last decade, and ELT > pure distributed processing in terms of both maintenance and performance [in most real-world cases]). Having good knowledge of both file store/Spark and data warehouse/SQL pairings is pretty much all you need if you are gunning to be a technically proficient manager. I’d argue against spending too much time with Hadoop proper. It’s antiquated, slow, breaks all the god damn time, and IMO these days is a red flag that a company’s data team is behind the times. Worth knowing the high-level contours of Hadoop though, as some places live and die by EMR. Also, mentioned this above, but strongly recommend moving yourself out of Big Query. Yes it’s light in maintenance, but not having experience or knowledge in the mechanisms of a data warehouse only hurt you.
Thank you for detailed response and no trolling. I d say I am pretty good with overall software engineering may be that’s why I got into data engineering without having any prior knowledge and in six months build a good rappo within team. I understand no one cares about hadoop now and I should pretty much step up my knowledge in SQL and pyspark. I want to follow up on your last part you mentioned move out of BigQuery? Is that because I should first understand Data Warehousing ? Or did you mean something else?
Because Big query is what I am using these days on GCP and consider it similar to sql obviously with specific syntax.