How to transition into ML Engineering from MLOps?
I am a mid level Software/ML Ops Engineer. In my previous company, I worked on a team that productionized ML models. They were heavily sprint based. The data scientists built the cool models and took ownership of solving the overall use case/problem. My interests have always leaned towards the algorithm and ML model side of things, due to various reasons, I wasn’t able to do much courses or side projects in data science during my previous job. I’m now looking to transition into ML Engineering roles where I get to train new models and develop new algorithms(pretty much open to any area, preferably NLP/Gen AI). What’s also appealing to me is that data science & ML (model prototying, training etc) itself doesn’t lend itself that well to weekly sprints, due to the nature of the job and tend to go with monthly milestones or updates which I’m more comfortable with. (Correct me if you’ve had a different experience in your company). I had a background in machine learning from before, so I am comfortable with the math and classical ML, CNNs and NLP DL basics like LSTMs etc 1) What is the best way to make this transition in the next few months to an year? I’m more likely to land another MLOps or Infra role in the next few months, but I want to make this transition sooner than later. 2) Thinking of starting with Kaggle projects and revising some basics. Not sure if Kaggle projects will actually help me land a job. If you have any other ideas, I’m all ears. 3) What are some things that surprised you after transitioning into ML Engineer after being a software engineer? Any tips or gotchas? #data science #tech #machinelearning #machinelearningengineer
Following
Haha. I’m the opposite. I’ve been an MLE and DS for 7 years and now doing courses and gauging opportunities to switch to MLOps and Infra teams internally. I feel like it’s easier to make your switch than my switch if you do it internally.
Fair point. Curious to know why you’re switching to MLOps ?
I’m in general infra, also want switch to ml infra
Dude you picked the worst possible time. In order to do what you wanted a year ago, you needed to tenure in FAANG and a have prestigious MSCS at the least. Generative AI/NLP and deeplearning is the most competitive field in the CS world period. Now that chatGPT and bard are picking up, it's only going to get wayy harder from here
I know the market is tough and there’s a lot of competition, but that has always been the case. The rise of ChatGPT has also spurred a lot of job creation in the Generative AI space.
Honestly OP, your current skill set is more rare and more appreciated, especially in the current market. If you like it but just feel like switching, may be reconsider … grow your skills to the next level in MLI and you’ll be set for good. Check out all those companies who removed DS and MLE positions from their careers page, yet still looking for more ML infra and Data Eng folks
What do you think about the future of MLOps in the next 5-7 years? I feel like it’s just going to come to a point of maturity at which stage it’s just going to br heavy on maintenance like a devops rolr
In my experience MLops and ML engineering are used interchangeably. Both rely more on software engineering skills and less on an understanding of statistics or even applied modeling. Things like docker, ci/cd, git/GitHub actions, knowledge of how to use aws/gcp to train, deploy models, host model endpoints, task orchestration tools including like airflow, metaflow, kubernetes etc along with data engineering skills are usually what the job entails. The advantage here is that most quantitatively minded people don’t really enjoy this and prefer to focus on statistics, so there may be more opportunity (or less competition) if you don’t have an ML phd. ML researchers tend to find the above tedious and not intellectually rewarding.
Wouldnt ml be more about models and algos. I would rather work in mlops and ml platform
I think the difference between a ML engineer and MLOPs is quite close and depends on company. I think you might be interested in checking out what other ML engineers are building: machinelearningatscale.com
I am regular SWE and want to get into mlops. What should I do? Thank you.
I would not suggest.
Why not? Regular SWE roles are boring. Mlops sounds interesting.