Is DS filled with many more expectations than SWE? Communication, presentation, dealing with conflicts, SQL, Snowflake, Teradata, Tableau, R, Python, SAS, packages (TF, Scikit Learn, Pandas, Numpy, Matplotlib, Seaborn), design (for decks), higher level literacy, Office, stats (endless here), ML/AI, algorithms, linear algebra, Hadoop, Matlab, Unix, Git, Jupyter, LaTex, Jira, Spark, etc. God forbid you also need some business knowledge. I just feel that at every interview I am asked something new. I am jealous of SWEs who need to practice algorithms, one language, LC, and system design. Do I miss something? Are there indeed more expectations from us (DS) than from other areas, or I am just completely ignorant about SWE?
Yes. That's spot on.
Which JD asks for all that?
DS requirements are very company and team specific. Also the packages you mentioned are basic ones that everyone uses. No one expects you to know the functions within rather you are required to know which would be helpful. E.g. data manipulation->Numpy, pandas. Model building -sklearn. For tensorflow you are required to know how basic operations work like convolutions, max pools etc. these are generic across multiple platforms and are derived from the functions of DL architectures rather than specific platforms. Production frames are used by teams that model and deploy services. Not many DS teams do that. Even if they do it is rarely at the level of SWE. FYI, DS salaries are comparable to SWE. Unless the role actually is data analyst you would be paid handsomely. You have to focus on the math more than the code
If you read this http://www.hdip-data-analytics.com/_media/resources/pdf/s4/build_a_career_in_data_science_v2.pdf (section talk about companies) you can see why. There ML/DL stack is quite chaotic and understandably so. There is nothing standard about the interview areas. But you can DM me for some personal notes. Don't want to put it here because of self-bragging.
Wow, thank you for the book! Curious to read it.
Yes, SWE requirements are way more manageable. DS sucks. You also correctly noted stats knowledge requirements can be crazy. There's A/B testing, causal inference, random statistical models, probability etc that you need to know to reliably pass interviews. The problem is there's no good way to prepare for these to consistently do well in interviews other than work at a company for a several years.
Yeah it’s the other way for me, I’m trained in stats so this stuff is second nature to me. It’s the data structures and algos that I need to work on.
DM me, same boat
Yeah, DS is a tougher field than SWE. If you want $$$ and haven't already done a PhD in a quantitative field, go for SWE. There's no place where this is more apparent than for MLEs. MLEs will often go through exactly the same interview process as SWE, but then have additional expectations specific to ML. They then get paid the same. And MLE is one of the "hot" areas in DS.
It's a dog-eat-dog world out there - you also need experience of putting code in production. These interviews are simply fishing for you to fail rather than pass - it doesn't have to be this difficult because what you're asked in the interviews is not necessarily done at work.
SWE don’t need to know one language. They need to know one language well enough that they prove that they could work in any language. And wtf do you think system design is lol? Just some quick easy thing?
Personal Finance
23h
2942
Calculated my NW and now i can't sleep
Tech Industry
7h
2166
Crossed a line with my boss
Software Engineering Career
10h
1256
Cleared Amazon onsite, but lowballed.
Tech Industry
11h
843
Update: Trans Coworker Stealing Breast Milk
Tech Industry
8h
1383
I am starting to think Chinese interviewers currently fail non-Chinese candidates on purpose.
SWE is a way better field to be in than DS. The hotter the field the lower the requirements for the same pay. Go try finding a 150k job in biology and let me know what happens.