Best Order to Study ML/DL?
Planning on starting OMSCS for an eventual change to DS/MLE adjacent roles in January and want to make the best use of my 4 month break. I’m from a non-tech engineering background and fairly confident in my math skills and have a solid exposure to python. I see a ton of books recommended and recommendations to learn ML first, or DL first and even broader AI first. I tried coursera video courses and low key hate them. So basically is it better to start with DL and use the deep learning book, or start with ML and something like ISLR/PRML/Probabilistic ML or look at more general AI like in Artificial Intelligence a modern approach? A generalized best plan of attack and some tips on how to make the best use of studying would be fantastic! TC 110k
Autonomous vehicles you have RL, control systems, vision, sensors and actuators and fair amount of physics models? But yeah NLP isn't probably a big area for this area. In general vision, sequence based prediction models would be of interest for you. A very useful but rarely used model in ML/DS is estimation models like Kalman filters, particle filters.. I'd keep that in low end of list but will be a bonus point for some recruiters ( if they understand how useful these can be) . Stats is an ocean- the big value of statistics is in helping somebody understand data and it's limitations, I don't know how useful stats is for vision but it's great for tabular data. You can check stats from "design of experiments" related books, it probably gives you a good set of working skills
There's probably so many different subdomains within CAV that trying to nail them all would be a pursuit of madness. I think some form of RL/Systems/Vision would be as close as I can get. Pretty much I'm limited to what OMSCS has to offer which seems to be a surprising amount from what I can tell. Initially I got into the UIUC online CS masters for the fall but they had a seemingly low amount of actual good statistical learning classes. Oddly enough the AI4R class there does cover Kalman/Particle Filters, PID and SLAM and thats what I'm planning on taking my first semester. Stats is almost terrifyingly huge, I've been flipping through all of statistics to get an overview of what I dont know and then intro to probability course to get back up to speed there, will check out some experimental design texts!
Nice to see civil engineer get into DL
Thanks! It was a weird journey to get here. I loved studying civil engineering but hated the actual work. Drawing in CAD all day was not my cup of tea and give it an honest shot for like 5 years. I thought I wanted to do something more innovative and tech-y for better pay and thought backend was the move as I thought it was a lower hurdle for me to climb without going back to school. In the end I felt like ML/DL weren't even things I should consider, but looking more into it I realized that it's way more interesting than everything else I saw in tech and that my real passion is using fun and interesting math to solve problems.
Omcs I'd focus on doing courses that you can graduate quickly. You'd be better served learning topics from the best courses that teach you that. In omcs it might be better to focus on learning computing systems, embedded systems, networking, database design etc that will give a solid CS foundation, which will help you become a better engineer long term. In civil equivalent, learn structural engineering , geotech, basics of transport, environmental, project management and engineering economic, everything else can be derived from these. Also one BIG area you can bring to the table that will differentiate you is Operation Research. If you can brush up LP, MILP and NLP . Its mind blowing how Data science courses have scant regards for Operation research! It's so rich in methods and applications with proven mathematical guarantee. I can't emphasise how useful OR is when you need to make business critical models.
Interesting! I was always under the assumption of that it would arguably be easier to self-teach things like database system and others but focus my time in grad school or more of core ML classes and some of the better systems classes like OS and networks. I get the comparison, in undergrad we all take the breadth to decide what we like, but in my masters in civil I deep dove down in transportation. I take it you were a civil as well and made the jump to ML? Enough about me, I’d love to hear more about your experiences within civil and how you escaped and your career afterwards!
Don't study broader AI it's used sparingly Here is a game plan 1. Linear algebra - nail down Matrix, vector operations, Matrix inversion to solve equations, eigenvalues, eigenvectors, Matrix decomposition - svd, PCA. 2. Probability and statistics - learn about basic tests like binomial, Dickey fuller etc to get some basics of understanding data which 99.99% of ML from CS folks lack. It helps you a lot in validation of results. Probability you need to understand basics of distribution, how various measures like mean, median , skew, percentile are used to describe them DL vs ML really depends on which industry and what kind of technology are you looking at, lot of business applications like finance, marketing etc you deal with lot of tabular data - various ML techniques specially boosted trees win over DL For NLP, vision DL wins, for graphs it's a melange of techniques from spectral algebra, CS , GNN are still not super dependable For ML hastie books For DL Stanford CS 22x series is the best
Appreciate the help and that makes sense! I got linear algebra mostly down, just took it at a CC this past semester since I literally took it over a decade ago! Didn’t touch svd or PCA though. But will make sure I do! My stats knowledge is pretty skewed, took a basic intro to stats class in undergrad but then a few different grad classes that covered linear regression, binomial regression/zero inflated, logistic regression etc. I don’t remember much of it though. Any recommendations for stats books? I’d love to do work within the connected and autonomous vehicle or traffic analytics industry as my domain knowledge is traffic engineering, but tbh I won’t be picky with industry focus if someone’s willing to hire me.