I like
Hermione, but I love the HP so it doesn't bug me one bit that
Hermione is associated with HP. However, I do think the associations are a little too HP witchy as a first name. But if you want to use it, go ahead. I would stay away from other Hp names though, like
Poppy and
Penelope. You'll pretty much have to stay away from all flower names there is too, because JK Rowling has used most of them. But not
Violet, I think?