The list of skills this field requires seems never-ending. To name just a few, one must be proficient in Python, machine learning, SQL, data analysis, GCP, Azure, AWS, advanced Excel, Tableau, statistical analysis, Spark, ML Flow, Docker, parallelization, Airflow, and now Gen AI. I mean, machine learning on its own is a complex field. I do admit I'm relatively new to this field with just over a year of experience but the level of expectation to be familiar with all these technologies and concepts is frustrating and overwhelming. How does one even deal with this? And don't even get me started on job interviews—what exactly are they expecting from me? #data #dataanalytics #datascience
My opinion - some of these are basic skills you should definitely have, some of them are more job specific. But Data Scientist is not an entry level position so yes it has a lot of required skills. Advanced Excel to most people is a pivot table or an XLOOKUP so you should be able to do this regardless. SQL, Python, and “data analysis”/statistical analysis are pretty fundamental for the job so yes you should be skilled at these, and very skilled in at least one of SQL/Python. Tableau is one of the most common data visualization tools for building and sharing internal dashboards so yes, knowing how to work with it is expected. Machine Learning is very broad, and not all DS jobs will work with this but yes you should know about the basic concepts, their trade offs, different models, and how to implement them to some extent. Then you have a bunch of tools listed which are fairly company and role specific. You won’t need ML Flow if a company doesn’t use Databricks. You don’t need Azure if they use AWS. The VAST majority of DS roles will never touch Gen AI. I’ve been working in analytics/DS for 4 years and have never had to create a docker container, and have only built a few Airflow DAGs because I wanted to learn about it. Be good at the basics: SQL, Python, (sadly) Excel, and statistics/analytical methods Be aware of the tools, pick 1 to be reasonably good at.
YMMV - I went from a “sql and Python is enough” role to my boss casually dropping two new tools I should learn in every 1:1. I cringe when I hear the phrase “just spin up a …” now
It’s a numbers game for anyone. Prior to oracle, my most interviews were lc and I failed those. Oracle was the only one where coding was implement ml algo, one pandas question and other rounds were past projects or ml concepts I was in RIF and am interviewing again. So far my interviews have been conversationally talking about my past projects and it’s weird, coz I won’t be able to filter out bs’ers in this format. I have seen many smooth bs’ers who can’t implement a mean square loss
You can't bs if you didn't actually do the work. You can talk high level, but won't be able to elaborate on details, showing you are a bs'er.
You want to practice hundreds of leetcode instead? Be my guest.
Who said I’m not already?
That is awesome
All this work and SDE’s get paid more with lesser YOE on average lol
Cause most businesses need SDEs to make things work. DS is more of cosmetics, it’s a need not a want for most
Also SWEs usually have a lot of these skills plus others which are harder in different ways. SQL and advanced language skills (python include) are the usual norm. Same with using Docker, cloud tech, and others. True SWE doesn’t use MLFlow or other ml specific platforms but they do use other equivalents for their roles. I came from a DS/ML background and transitioned to SWE and can say at least from my experience SWEs generally need to know more overall technologies and languages.
You forgot to mention statistics, probability, Deep learning
Data science is not machine learning. It contains parts of ML but stuff like Docker is mostly the domain of MLE.
Well, you’re making it sound hard than what it really is. If you know aws, no one is expecting you to have azure knowledge. Same goes for excel, python, tableau and most of the other stack you listed.
None of the list is hard tbh. Especially with all those packages… I use half of them and I’m not even a DS. Maybe tech is too hard for ya
You’re so right man. Gosh. You should have a podcast!
Depends on company to company. Generally the smaller the company is the more hats your expected to wear which means more involved with the non ML side of things. The bigger and more established a company’s ML team generally means you get to focus more on the our ML side of things since the responsibilities/process/infrastructure is built out already.
You really just need to know SQL and a little bit of Python.
I wish that were the case; it doesn't seem like it.
Heck no... You need to ml, little DevOps, DL, little swe