Coding for Data Engineer roles.
L6 YOE:12 TC:$265k A bit about my background. I am a tenured(5+ years) Sr Data Engineer with Amazon. Although, I started as a SDE(not with Amazon), I made a switch into Data Warehousing very early in my career(within first 2 years) and have been a Data Analyst/Engineer ever since. Mainly working with different flavors of SQL and orchestration/ETL tools built by internal platform teams. Over last couple years, as a senior engineer, my D2D is mostly 20% hands-on, 20% mentorship/recruitment, 30% solution architecture and 30% reviewing other's work + data strategy/governance etc. Only recently, I have started working on modern DE stack. Python/Scala and Hadoop, EMR, Airflow/Flink etc. Over last year, I am exploring Data Engineer roles with other companies and have appeared for tech screens with a few. What I have experienced is majority of the top companies(in terms of pay) have coding heavy tech screen(which used to be only SQL back in the day). Even still, my current team only screens for SQL if it's a Data Engineer role. I have bombed coding part of each and every one of those tech screen miserably and cleared SQL part with flying colors. And yes, I did learn DS and Algos back in college; but it was like 13-15 years ago. After failing those tech screens, I have developed this fear that I am not good enough with coding and I will never be able to join another top company to get a massive jump on my TC. And eventually my skillset will become obsolete. I know upskilling and practice is the only way but every time I start leetcoding ds/algos, I just loose my ability to think and go blank. This has started to affect my overall confidence and mental health. I figured there must be other data engineers in similar situation. I wanted to reach out to you'll and understand if anyone was in this situation and how did you get out of it? Alternatively, are there still any companies out there that do not screen coding at all or atleast not at the scale of SDEs when they interview Data Engineers? Any feedback from HMs for data engineer roles will also be appreciated. #data #dataengineer #dataengineerinterview dataengineerinterview #tech
Following
This is independent of data engg/SQueaL background. Interviewing/competitive programming are completely different and isolated skillsets to SDE. You become good at it with tools like interviewcake and codeforces.
Teaparty - are you hiring for data engineers in your team?
Google explained it very clearly above. Adding to what Google said: AFAIK, originally DE role came to market to represent SWEs with expertise in building and managing Data Systems. But Amazon, Facebook, Google rather picked only the Analyst and Analytics part of the DE scope, eliminating or minimizing the engineering aspect of the role, turning them more like some combination of traditional Database developer, BIE, BA, Data Analyst with some scripting skills. But in Amazon there are definitely a lot of DE teams who work more on the engineering side (scope and development style are almost same as SDE), while BIEs manage the SQL, Datanet, Dashboard aspect. So, unless your interests are more on the Analytics and traditional DW and DBMS side (which will not pay same as SWEs), you must revisit some of your CS fundamentals on DS and A, LC practicing, Software Development processes (CI CD, build, Infra as Code etc.), System Design (data system design), Design patterns (at least the most important ones), writing modular code (properly designed OOP oriented code) etc. to get a job as DE in other companies which pay their DEs and SWEs equally (most companies out there do that).
Every company screens SDE level coding for DE these days. In Amazon, google and Facebook data engineers are basically analysts with an elevated skill set. In almost every other company they are software engineers The best bet is to really grind CS fundamentals For a year and leave. I’m facing the same problem in goggle where a DE is an analyst. However I had prior experience as a software oriented DE and Imclose to few offers. Don’t hesitate to leave Amazon. FAANG SWE != FAANG DE However Netflix and apple I hear are SWE oriented DEs Just as an example - I got asked “count the length of the longest palindrome in a string” in an interview recently which qualifies as a leetcode medium and had to do it it less than ten minutes
How’s google interviews for DE’s like? Similar to SWE?
No - it’s a data analyst role so all SQL and product sense