In between you and the person or situation, there is a mental image in YOUR mind which YOU have shown to yourself.
Initially you think that the person or situation is causing the fear you're experiencing. You think that the "fearful thing" is inside that person or situation, or coming from them. Like they're causing it.
It's not true at all. They are not doing anything to you!
Somewhere along the lines, YOU came to BELIEVE IN FEAR, and YOU put in your awareness a mental image of something scary, and now you're experiencing that visualization simultaneously, projected on top of the people or situations outside of you.
The fear is really you scaring yourself, because YOU believe in and are imagining, visualizing, a scary situation.
When scared you will try to fight with external people or situations as though if you fend them off enough they'll stop scaring you. But notice instead, no matter how much you try to "un-scare" yourself it won't generally work, because YOU are actually still believing in the fear. YOU are still generating the fearful image in your own mind. It's coming from YOU. Changing the person or situation will not undo what YOU are doing to yourself!
You have to realize you're literally doing this to yourself. You are generating this mental image, really just an idea, and are producing fear internally. You're nuts for believing that anyone else is causing it.
Until you can see that it literally is coming from you and has nothing to do with what's out there, and you decide to STOP BELIEVING IN whatever scary thing you were believing in, it will remain.
It's only because you believe in "real attackers" that you are scared. It's only because you believe in "real sin" or "real guilt" or "someone's really going to come and punish or hurt me" that you feel scared. It's not because they ARE, and it's not because they're even involved AT ALL.
It's purely your own belief. You are choosing to perceive fearfully and your fearful perceiving is putting a fearful vision in your mind and you're experiencing it like it's real because YOU are making it REAL.
Seeing this, you see the pointlessness of believing that someone else is actually the cause. Owning up to being the cause, you can finally see that you yourself are scaring yourself, you are the ONLY SOURCE of this fear, and therefore you can simply choose to stop believing in it. Stop believing that you need to be afraid of something "out there". YOU are the one terrorizing yourself.
It also frees you from the MYTH that you have reasons to be afraid that are beyond your control, ie that other people ALSO have the power to "cause you" to be afraid. They don't. There is no power over you other than your own, and there are no causes outside of you at all. So if you'll stop attacking yourself with fear, there won't be ANY other sources of it.
If you can then choose to stop having this belief, upon recognizing that you were the one making the fear, the fear will disappear. Illusions recognized as illusions must disappear. The fear will vanish and you will be restored to safety and confidence.
Your assertiveness will assert that there IS NOT a reason to be afraid, choosing that this is the truth, rather than WAITING for some other person or situation to stop "making you afraid".
I fall for this trap often. It's the basic ego trap of believing fear is reality. It can take a while to admit to myself that I'm the cause of this fear and to see that it is literally coming from my own believing and my own choosing. But then once I see it, I can realize the folly of my own self-attack and simply choose to no longer do this to myself.
Who can scare you if the only one scaring you EVER is yourself, and you decide not to?