get the better of me
When someone or something defeats you or overcomes you. You can get the better of someone in an argument or a fight, but, as in this case, one of your own impulses can overcome your judgment and get the better of you.