40% of work is reverse engineering database structure all the way back to value input in a wide array of applications. Business rules, native application data types and validation and the entire methodology for field input. How the fields get copy pastad by users and changed through software stack, and advising best source for data points. Other 40% is SQL development and delivering custom APIs to software development teams so they don't have to understand the application/ database 'context' to build software. Building new data structures, tables and engineering candidate keys to tie together applications that were not engineered with proper PK/FK relationships(little to no system to system automation) Last 20% is prototyping new databases, structures, enriched tables and tableau reports that requesters didn't even think was possible. Old cable company, super immature at using data for software, (field formats and preserving hierarchies matter) to the point of screaming about why 100 variations of a customer name obliterates any chance of customer profiling let alone automation and ML. Current title - Network Engineer. HR has 0 idea what to do with me What is this job in other companies or does it not exist? I have no idea what I should be getting paid or what title to ask for.
Kind of. Learn python on your own outside of work with a bit of AWS and PySpark and you can up your TC greatly I imagine, probably with a better company too.
We do literally everything onnprem, they are vehemently opposed to any data going to the cloud. I assume you are referring to the AWS free tier stuff though for self learning?
Good to get experience outside of work. I didn’t get exposed to aws in my last 2 jobs. I practiced outside of work. I typically suck up some of the free credits, but unless you are leaving clusters or instances on24/7, it’s surprisingly cheap if you are sure to monitor everything properly and not forget to turn off an instance or cluster. If you know python check out boto3, the python sdk for aws. Super awesome. If you are in data engineering, just focus on EC2, EMR, airflow (python library) for automated workflow/etl, and PySpark to be used on EMR. Eventually get to Kafka/kinesis and maybe flask api/microservices
data engineer
Even though our tech stack is kiddie style? I cant get IT to install anything Java/ Python related so I have to do all cleans and transforms in SQL. It destroys my resume for other data engineering roles at more mature orgs to not have it in practical work experience :(
Roles and tech stacks are mutually exclusive, a SWE who only knows jQuery is still a SWE...kind of.