If someone impugns you or your honor (in the sense of attacking your “good name” or reputation) ignore him.
If that isn’t possible, seek a graceful, even a humorous way out — like the American in France who once met a Frenchman’s challenge to a duel by selecting for the weapons “apple pie at ten paces.”
After all, what is that false “honor” but an affirmation of ego: the root cause of all suffering?!
Suffice it if you, following your own conscience, act honorably. People’s unflattering opinions of you are not your problem, but theirs.
From Living Wisely, Living Well by Swami Kriyananda