How important Data structures are for the role of Data engineer?

Intuit
gamersa

Go to company page Intuit

gamersa
Mar 30, 2021 95 Comments

Been working as Data engineer from past 5 years, never used Data structures in my projects. Many of interviews has data structures as 1st round of screening. Does it really make sense?

Is data structures really required for data engineer?

Edit 1: we have many distributed processing framework like spark which process data in different nodes together. I assume using python legacy data structures into the code, pushes all data into single driver..
Yes we do use list, tupples, dictionary in python wherever it's required. To store minimal information.

We never use python legacy code to read write or to process data. It's pretty outdated right??


#data #dataanalytics

841 PARTICIPANTS SELECT ONLY ONE ANSWER
VOTE VIEW RESULT
95

comments

Want to comment? LOG IN or SIGN UP
  1. TOP
  2. RECENT
  3. OLD
 TOP 95 Comments
  • They both incorporate the word “Data”. Definitely important
    Mar 30, 2021 5
    • Amazon
      LpOIv56s

      Go to company page Amazon

      LpOIv56s
      That's why I always quiz java engineers with javascript questions.
      Mar 31, 2021
    • New / R&D
      king_james

      New R&D

       king_james
      As important as asking amazon engineers about amazon deforestation
      Apr 10, 2021
  • Google
    drakez

    Go to company page Google

    drakez
    Do you mean Leetcode type questions?
    Mar 30, 2021 8
  • Asurion
    tokkale

    Go to company page Asurion

    tokkale
    It used to be never useful earlier but, now data engineering is using software engineering principles, concepts. OOPs is there for python and scala. So data structures make sense to be something that DEs need to know.
    Mar 30, 2021 0
  • Amazon
    itsobvious

    Go to company page Amazon

    itsobvious
    I have been a data enginner for almost 4 years, I've never needed anything more than a hash map, list and strings.
    Mar 30, 2021 2
  • Data engineers do 70% work that software engineer do except front end part . Programming is must so the data structures. Without data structures how you are going to process huge n complex datasets? They are must
    Mar 30, 2021 3
    • Those are data structures
      Mar 30, 2021
    • Yeah, they need to understand the data structures available out of the box, their time complexities, behaviors. Which most people do. But they don't necessarily need to be able to print left bottom diagonal view of a binary tree in an interview. If they really did need to implement a tree, they can google it, or even read up Cormen if you are averse to Googling.
      Mar 31, 2021