When you're looking for another job do you set this on your linkedin profile?
Everyone should do this
Never set it to not open and then nothing visibly changes when you start looking
Keep it always on. Technically, you are always open: It just depends on the opportunity.
Does having it set to "Open" actually make a difference in the number of recruiters that ping you for opportunities?
Yep
Yes
Yes, though when contacted by a recruiter asking if I’m looking, I say I am passively looking and open to opportunities.