I have 13+ years of experience in Data stage ETL development, web automation testing, micro service development and now in Data Engineering. None of these shifts in technology/roles were planned, was doing what the personal situation demands. I was not goal oriented before but I was sincere, responsible and fast learner at work. None of these is helping me to grow in the career ladder. I want a stable career path forward. Should I move back to micro service development or learn AI/ML ?
Pursuing AI/ML may require a longer timeframe to become productive for growth or to become strong enough to land in good opportunities. This depends on your current seniority and whether you are willing to start over as a junior engineer or transition at a similar level. I still believe that data engineering is promising, esp if you have a background in microservice development. (You can likely leverage your existing network there assuming you know some folks in that area). At the same time learning AI is also a good option, since you said you have been adaptive which is a great skill for growth. However it's important to be persistent and be prepared to face setbacks while still maintain focus if you choose the path of AI. It very likely will test your patience :) Good luck!