I find data engineering pretty interesting and fun. You don't need to write complex distributed services or web-apps but mostly have to make use of existing data infrastructure and systems to build pipelines. However, it seems to be a relatively new field and maybe the market pay is also less than backed engineer or other software engineering roles - Data Scientist, Machine Learning Engineer, etc.. Is data engineering looked down upon, in comparison to other swe roles? What are the career prospects?
Nah, it's much more than babysitting. It's usually ETL - Extract Transform and Load. Each of these phases can be quite complex and if done right, can save millions of dollars. Once the pipeline is built, you can sit back and relax - this can be the beauty or ugly of it - depending on your perspective.
Aug 23 0