Confusing legacy codebase, totally stuck, no one can help, what do I do?

HPE / Eng brotein
Jun 25, 2018 37 Comments

I'm tasked with a complicated bug fix within a massive legacy codebase that is dependency hell with undocumented spaghetti code. I've been debugging an issue for a few days and have absolutely no idea what to do next. I don't understand the code and no one else does either. I asked my lead what to do and he said to set breakpoints and read the code. I asked where. He said doesn't know.

I've been systematically debugging the issue using every technique I could find on Stack Overflow and books like Code Complete/Debugging. Logs doesn't indicate any useful information. There are no error messages or exceptions. There are no unit tests. All the domain experts I could possibly ask have left. Basically there's no one to help, reading the code and debugging isn't giving me any meaningful data points. I have a general idea what's wrong and where it's happening but my mental model of the application is still weak and things just aren't clicking. I'm super frustrated and feel stuck. I'm also in the middle of a job search and feel like a totally incompetent dev who deserves to rot here for life.

TC: 95k. YOE: 2

comments

Want to comment? LOG IN or SIGN UP
TOP 37 Comments
  • McAfee JohnMcPee
    You are a competent dev that is in the wrong place. Of course all of this is not revealed during interviewing. The only way out of this is to get out of that team or that company.

    With that much technical debt, it only gets worse from here onward.
    Jun 25, 2018 1
    • HPE / Eng brotein
      OP
      Thanks, will keep this mind. I was thinking of jumping into another team if my job hunt isn't successful, but I'll probably keep leetcoding...
      Jun 25, 2018
  • Illumina Illumina
    The most important thing to do is understand what the code is supposed to do. If you don’t understand that, it’s hopeless. After that, put some print statements into the code so you can see order of events, and what’s happening.

    I was once asked to review a bug fix to the worst legacy code I’ve ever seen. There were goto statements all over. Dozens of bugs had been entered against it for over a decade. I rejected the fix, because no one could say with certainty that there were no other bugs. So I decided to spend a day writing down all the code paths and developing an idea of what was supposed to happen. At that point, I threw the original code in the trash and rewrote a critical function from scratch in a clear and concise way. The next year I was on the project, there was never again an issue. Moral of the story: sometimes the second step should be to throw some part of the old code in the garbage and start from scratch. (The first step being understanding the task at hand.)
    Jun 25, 2018 5
    • HPE / Eng brotein
      OP
      Thank you, I appreciate the insight.
      Jun 25, 2018
    • Amazon / Eng TimeOffCop
      This. I recently was working in a piece of code that didn’t make much sense. Then I tried to look at the big picture and suddenly I could understand the code and also propose a much simpler solution.
      Jun 25, 2018
    • Amazon / Eng ᕕ( ᐛ )ᕗ
      Bonus points if you wrote tests against the old code to validate your understanding of it, then used the same tests to validate your new code.
      Jun 25, 2018
    • Illumina Illumina
      Tests are great if possible. Unit tests can be really hard on terrible code though. At least implement tests for the new code you write.
      Jun 25, 2018
    • Capital One / Eng Cheeter
      Another path is to gather all requirements first, write unit tests, then write the code. Difficult to do for a monolith, but if you write in components, it will get you there quicker. And the added bonus that you’re now the expert.
      Jul 22, 2018
  • Oracle / Other nogo
    Are you telling me there is no senior in your team?

    Given you have only 2 years of experience I doubt that. No matter what if you are stuck always and always ask for help!

    If your lead is not helping escalate to your manager. It's his job to help!

    Also you can chk the history of file for write and reach out to guys who would have worked on it.
    Jun 25, 2018 0
  • HPE / Eng brotein
    OP
    Update: someone was working on a related issue and their code change somehow propagated and fixed my problem. I have no idea how this happened, but will need to review their code. Also thank you everyone for help, I wrote down all of these suggestions and will be using them in the future.
    Jun 28, 2018 2
    • Oracle / Other nogo
      Good for you. But definitely find the root cause. It will be a good learning experience
      Jun 28, 2018
    • New / Eng
      stitch

      New Eng

      PRE
      Raytheon, BAE Systems
      stitchmore
      I’ve had that happen. Definitely check out what they did and try to make sense out of it.
      Jun 28, 2018
  • Microsoft / Eng
    lusername

    Microsoft Eng

    BIO
    Yeah, right.
    lusernamemore
    I've found that the challenge in working in legacy code is often not finding a fix for a problem, but not causing regressions with the fix you're making. Ensure that your lead is aware of the challenges your up against. If they're not sympathetic or aware then that's not a good environment
    Jun 25, 2018 1
  • New / Eng
    Newtech

    New Eng

    PRE
    500 Startups
    Newtechmore
    Keep leetcoding
    Jun 25, 2018 1
    • HPE / Eng brotein
      OP
      Probably gonna go with this.
      Jun 25, 2018
  • Microsoft / Eng aJb52Ys
    This doesn’t make you incompetent.

    Whoever wrote this system with no tests is incompetent.
    Jun 25, 2018 0
  • Oscar / Eng
    tc/gtfo

    Oscar Eng

    BIO
    Hi. I am real and I am pretty friendly. Just here for some happiness.
    tc/gtfomore
    Leave.
    Jun 25, 2018 0
  • New / Eng
    stitch

    New Eng

    PRE
    Raytheon, BAE Systems
    stitchmore
    Do not, I repeat do not change any code if you don’t understand what it does and what it’s supposed to do (other than adding print statements or comments).
    Jun 27, 2018 2
    • Amazon max9chars
      Yes, just bow to the overlords that came before you because you are just too dumb to maintain code of this complexity. In fact, we should have been paid year-over-year just to stick around and maintain this complex puzzle that we created to show the world how smart we are, even though we had burnt out and stopped adding any more value to the organization. Now the Man will realize who he undervalued for years and years.
      Jun 27, 2018
    • New / Eng
      stitch

      New Eng

      PRE
      Raytheon, BAE Systems
      stitchmore
      Not saying you shouldn’t change it. You just need to understand it first.
      Jun 27, 2018
  • Intuit / Eng gCTT61
    Run away from the company if you are doing this day in day out. Understand that most of the companies where you will work WILL HAVE legacy code. You are learning a valuable skill.

    Here are few tips ( most of them were already mentioned in other posts)
    - understand higher level purpose of the code/class
    - draw out the flow on paper
    - Draw Class diagram using ide to see dependencies
    - step through the code
    - keep writing comments for the parts of code which you have understood
    - add more logging
    - don’t worry about solving the bug till you can understand the code
    - only after above steps - try to reproduce the issue

    Forgot to mention, step away from the code and take a walk :)
    Jun 27, 2018 1
    • Amazon seeeker
      I did this once in a previous gig. Helped me survive until I rewrote the complex contraption into - wait for it - another complex contraption that only I could understand. Tormented became the tormentor. Meek ruled the earth. Justice!
      Jun 27, 2018
  • New / Eng
    stitch

    New Eng

    PRE
    Raytheon, BAE Systems
    stitchmore
    You’ve just described my entire job. Get another set of eyeballs looking at it with you, set breakpoints, step through the code and see what it’s doing, add some print statements. Eventually when you stare at it long enough, it will all start to make sense. Kind of like seeing the matrix.
    Jun 25, 2018 0
  • Amazon max9chars
    Rewrite it - leave - watch the next one throw up their hands in despair 😀 The cycle goes on...
    Jun 25, 2018 1
    • Oscar / Eng
      tc/gtfo

      Oscar Eng

      BIO
      Hi. I am real and I am pretty friendly. Just here for some happiness.
      tc/gtfomore
      So evil, but rewrite it is basically killing yourself twice.
      Jun 25, 2018
  • New / Eng
    H1bguy

    New Eng

    PRE
    500 Startups
    H1bguymore
    What doesn’t kill you make you stronger. I think it’s a great challenge for you to learn. Don’t run away.
    Other comments above are very insightful. Just want to add that: seek for help from an experienced engineer. He/she will help u learn much faster
    Jun 26, 2018 0
  • Uber pmo
    Get the hell out of hp is your best bet
    Jun 25, 2018 0
  • Microsoft / Eng Xizhi
    Sounds like FB infra? Op are you in FB lol
    Jun 25, 2018 1
    • HPE / Eng brotein
      OP
      Nope I'm in HPE unfortunately, but now I know FB infra is not where I wanna go to next
      Jun 25, 2018
  • Square / Eng nom🍔
    If nothing else works, take out the nukes. Log every single line and a "bug assertion" (a log that tells you if you're in the bug state or in a good state). Start with the highest level function you know has the bug, then drill down where you find out the bug state happens. Rinse and repeat.

    If you can't even find any function in which the wrong state happens or what the bug state is, contact previous employees and offer them bribes...
    Jun 26, 2018 0
  • Amazon / Eng Glued
    Yeah, you got pretty unlucky. Try to ask to be assigned to another task (assuming this bug isn’t an emergency) if you’re just getting started at HPE. Debugging issues in that type of code base is gonna suck regardless of how good you are. Find a new team/company if you think you’re gonna be working on that kind of stuff for a while
    Jun 25, 2018 0
  • Bloomberg
    Nextgo

    Bloomberg

    PRE
    Amazon
    Nextgomore
    Hey try your best, and keep job search
    Jun 25, 2018 0
  • Indeed / Eng RachelsPup
    The typical way would be to create unit tests for the system then reimplement what you have to breaking it apart as much as you can
    Jul 1, 2018 0
  • Motorola / Eng
    Solaire

    Motorola Eng

    BIO
    Praise the Sun!
    Solairemore
    Have you read the book “Working Effectively With Legacy Code”?
    Jun 29, 2018 0