Im such a giver. Im always concerned about her first. I think it comes from fears of inadequacy
You shouldn't feel about this because it's quite common. Men are conditioned to be givers. What I've learned, though, is that the hottest sex comes from a two-way energy of giving and taking. Women like to know that they're getting you off. Women want to be adequate as much as men do.
If you aren't taking during sex, you're doing yourself a disservice. You should be having sex because it feels good to you. There's nothing wrong with enjoying yourself or liking orgasm as a man. The man enjoying himself, and enjoying his orgasm, is every bit as important to good sex as the woman enjoying herself and her orgasm.
Of course women appreciate the "Ladies First" mentality, but sex becomes boring if you bring that mentality to sex every time. Women want to feel like sex goddesses like men want to feel like sex gods. Yes, I believe most women will be frustrated if you consistently ignore their needs and come before they have a chance to come; but most a lot more understanding than we give them credit for. They don't always want or need to be satisfied by a penis penetrating their vagina. Sometimes they do.
Be selfish in sex just once and see what happens! Ask for exactly what you want, what's hot to you. You may find it electrifies the relationship. The woman will feel proud, you'll get what you want; sex will be honest. If you sense that the woman wants to get off, ask her. I've experimented with this lately and have found if I do it, it's a lot easier to get my partner off because she's really turned on by the fact that I'm so turned on by what she's doing.