As Jonathan Edwards said, labels are necessary. Even as basic as labeling someone ‘Christian’ or ‘American’ conveys much important meaning without redundant detail or explanation.
But if labels are misused? They are frequently employed to immediately stop discourse — usually when the one using them is losing the debate — as when one is called ‘racist’ or ‘Calvinist.’
What people usually object to when they use these labels is a caricature of the thing, not the thing itself. Most are objecting to a stylized straw man, and easy target for those ignorant of the real issues.
When believers, for instance, object to ‘Calvinism’ because it teaches that people are robots and God created some people just to condemn them, who wouldn’t object? The Democrats accomplish the same thing by claiming Republicans are for dirty air and water.
It is very easy for us to create a caricature of our opponent, or of his position, and then attack that. It is usually much easier, and more successful.