I'm afraid it is noticeable to at least a segment of the population even here in the US. I mean, we know
Rene is male and
Renee is female, and we know
Desiree and
Estee and
Aimee are female.
Then again, a lot of people here have never seen the name
Esme on a male, and honestly I don't think they're likely to. I met a baby girl named
Esme at the park recently and although I know it's supposedly a masculine spelling, I can't say I batted an eye.
Besides, what about
Timothee and
Amedee? They break the rule in French... so it's not a hard and fast gendering rule anyways.
I prefer
Esme over Esmae and Esmay any day. Changing the ending just makes it confusing and distracting.