Ok, I'm just going to spit out an argument that has been on my mind:
Words are not bullets. The harm caused by offense is categorically different than any (any!) physical harm.
This is a thesis that needs to be defended, and at the same time qualified. Whenever people seek to enact speech codes they cite the harm that comes with offense. This makes perfect sense, because offense is a very real harm. Offense hurts the worst when it is applied systematically to people with very little power. The constant harassment of women and minorities in my discipline (philosophy) is an example of offense as a moral wrong.
That qualification in place, I want to draw as bright a line as possible between physical violence and verbal violence. I recognize that there is a lot of gray area here. Even John Stuart Mill admitted that it was one thing to say "Bread sellers are starvers of the poor" and to say the same thing, in front of an angry mob in front of a bakery.
I've got three reasons why physical violence is always worse than verbal offense. The first one is the most practical. Just as a matter of moral epistemology, we are better able to evaluate the harm caused by physical actions than by emotional offense. Doctors can tell us how hard you were hit, what bones were broken, and if you will live. Judging how offended someone is, really, is always guesswork.
The second argument is Rawlsian. Bodily health is a primary good. It is one that enables any other good you might seek after in your life. Harms to primary goods are always worse than harms to other things.
The final argument is the hardest for me to articulate, but I think it might ultimately be the most important. In the case of offense, person harmed has more control over the outcome than in the case of physical harm. This is not to say "sticks and stones can break my bones, but names can never hurt me." In fact, names can hurt a lot, especially when they are thrown consistently at people with very little power. After a while, even the most tough-minded person will feel the blows. Those of us with even weaker constitutions will crumble much more quickly.
That said, it is at least possible to shrug off the harm of an offense remark. You cannot decide not to let blood loss bother you. Names hurt, but we can mitigate that hurt by deciding not to let it bother us, and we are all a lot better off if we all agree to let some shit slide. Humanity's greatest strength is our ability to know. This is why things have basically been getting better for us for the last 35,000 years. Our ability to know depends on our ability to imagine all kinds of crazy shit. Even shit that seems to go against God.