How to respond calmly to someone trying to rush me on squat rack?

I did 5x5 squat then 5x5 overhead press on rack, 3 min rest in between each set

Guy asks me how many sets I have left half way through, I tell him 5 sets. I politely offer for him to jump in, he declines

I stretch in between sets, but my rest is timed on an app, so it doesn’t add to my rest period

When I have 1 set left, I let him know I’m wrapping up. Instead of thanking me, he gets passive aggressive saying “next time you should hurry up instead of stretching, it’s rude of you to keep others waiting, especially because you’re only doing 3 reps per set”

I responded that I am resting in between compound lifts, and that he shouldn’t be rushing people or else they might get injured, and to wait his turn like I did

It goes back and forth briefly with him getting offended, and me telling him that he’s the one that’s being disrespectful. I end up telling him sternly to shut the fuck up and get out of my face (without raising my voice), and that instead of doing my deadlifts elsewhere out of courtesy, I will do them here instead. He cowers in white pearl clutch fashion

I put back in my headphones and take my time finishing

I am usually in control of my emotions but this really set me off and I could’ve easily done something stupid like punched him in the face. He was an entitled brat but my reaction was unnecessary

What is the appropriate reaction to this? Basically, how do you deal with a Karen without resorting to putting them in their place?