Jem Black, Ward, Belcher and Mace were all bare knuckle fighters form the early 1900's. Jem Finer writes/wrote music for The Pouges. Jem Stansfield presents science for the BBC. It is most definitely a boys name being a shortening of Jeremy or, in earlier times James.
This can also be a short form of Jeremy, as in Harper Lee's novel To Kill a Mockingbird. The main character's brother is named Jeremy Atticus Finch, but he's called Jem.