ETL tools better or hard-coding?

New / Eng
DataMentor

NewEng

BIO
ETL developer, Talend, ODI, Power BI, Tableau, QlikView, SSIS, SCRUM, Hive, HQL, SQL,Hadoop,R,ERP:Quickbooks, Intacct, Netsuite, DynamicsNAV, SL,AX,GP, EBS,JDE
DataMentormore
Jul 5, 2018 6 Comments

Personnaly I use ETL tools at work. However, most of large industries settle on using coders to do the integration and transformation ? Isn't it easier and time efficient to use tools and dynamically manage the data flow than having to write codes in terms of cost vs time chart?

comments

Want to comment? LOG IN or SIGN UP
TOP 6 Comments
  • Amazon / Engdiddler
    i think most companies just aren’t aware of prebuilt ETL, the few that do know look down on them vs custom
    code.
    it’s like an intellectual superiority contest.
    at certain levels of scale ETL just doesn’t work effectively though, so maybe good to start ETL then once you out grow move pieces to custom coding
    Jul 5, 20182
    • New / Eng
      DataMentor

      NewEng

      BIO
      ETL developer, Talend, ODI, Power BI, Tableau, QlikView, SSIS, SCRUM, Hive, HQL, SQL,Hadoop,R,ERP:Quickbooks, Intacct, Netsuite, DynamicsNAV, SL,AX,GP, EBS,JDE
      DataMentormore
      OP
      Oh how so? I have used SSIS, ODI,Talend, Informatica and each one of these tools seem to have the capability to do exactly what a custom code can...however, if organizations have a question on handling larger volumes of data to move across a VPN, we can always create a looping package to extract data, which is fairly quick....i guess it comes down to having user knowing how to custom create codes within the tool which can be rather useful..
      Jul 6, 2018
    • Amazon / Engdiddler
      one thing is dynamic throttling (eg slow down writes based on health of system) - hard to do on etl, easier to do in code
      Jul 7, 2018
  • Amazon djdjie+$2)
    I think for the most part the pros of using a tool outweigh the cons.
    Jul 5, 20181
    • New / Eng
      DataMentor

      NewEng

      BIO
      ETL developer, Talend, ODI, Power BI, Tableau, QlikView, SSIS, SCRUM, Hive, HQL, SQL,Hadoop,R,ERP:Quickbooks, Intacct, Netsuite, DynamicsNAV, SL,AX,GP, EBS,JDE
      DataMentormore
      OP
      Definitely! It makes sense rather than have coders write custom python and JAVA codes, cause making adjustments and using reusable templates gets rather tedious on hard coding!
      Jul 6, 2018
  • WalmartLabs Happy?
    Never heard these names before. At many companies there is no good process to experiment with paid software that could be useful
    Jul 7, 20180

Join verified employees in our anonymous social network!Download the app!

close