I strongly believe that your clothes should help you achieve your goals, not inhibit you from reaching them.
That means that if your heel is too high and you opt out of a great adventure simply because your feet are killing you, your clothes aren't doing you any favors. Conversely, if you are convinced that comfort is your paramount goal, opportunities may pass you by.