Being Assertive

It is really hard for me to be assertive.
It stresses me out.  I have an actual physical reaction to it.
In front of my closest friends and closest family, I am able to assert my true feelings.  And yet, even then, being assertive is really difficult.  I usually break down crying after confronting somebody.
I hate being assertive.  I hate being assertive, and yet it stresses me out to reveal what I want and stand up for myself.
I'm not sure if it's a confidence issue.  I'm not sure if it's a social issue.  I don't know why I don't like being assertive, but being assertive always makes me anxious.