I love this name, but like you I live in the U.S. and I worry about the cultural/racial connotations of using the name. I used to think maybe it would be possible to get away with, but after a thread we had discussing this name fairly recently, I'm definitely not so sure. :-/
Here it is. It might give you more insight into how many people on this board (esp those of us in the U.S.) feel about using this name:
http://www.behindthename.com/bb/arcview.php?id=804642&board=baby
How about maybe just using
Jemma or
Gemma?