The Washington Post blog The Root has an African-American atheist, Mark Hatcher, saying what that’s like.
[One day] I’m walking across campus, and normally don’t have it on, but I had my Atheist t-shirt on. Somebody came up to me and said “Oh my God, I thought I was crazy, I thought I was the only one. Thank you for letting me know I’m not insane.” That’s understandable in our community. You gotta love Jesus. If you don’t love Jesus, you gotta love somebody. My mom’s first question to me was ‘What, so you don’t believe in anything?!” And that’s hard in the black community. You gotta believe in something in order to be a complete person. This person coming up to me, saying that they thought they were insane because of the type of pressure that was on them to believe in something that they just simply couldn’t, I was like, “You know what? We need a community here”…
There are other things you can believe in though. You can believe in a better future for humans. You can believe in hope, in solidarity, in compassion…you can even (though you will get a lot of people yelling at you) believe in progress. You can believe in music, in art, in love, in sex, in nature, in beauty – damn, you can believe in a lot of things. They don’t have to be a person, especially not a magical person.