I agree with you. The primary focus should be on helping customers. I think, by setting a reasonable rate, and using that as a cut-off for making decisions, you can reduce the amount of time you obsess about this aspect of your business, and focus more on improving your quality of work -- both production, administrative, and marketing.
But don't let customers guilt-trip you into working below your rates. Instead, refer them to someone who is HAPPY working at those lower rates.
If it's a really good cause you believe in, set aside some time for pro-bono work. This work can be purely personal, to help the world, or can be part of your marketing strategy, but pick your pro bono projects well, so that you don't get burned out.
I personally do volunteer writing for metaverse-related stuff (which I am a passionate believer about!) and for some causes I believe in (freedom of the press around the world, our local town library, other local community groups). But if someone asks me to do something for free that doesn't fall into my two pro-bono categories, or that takes more time than I have set aside for this work, I put them in touch with someone else who can help, instead.