Honestly.
Ashley you could get away with...but not
Ashlee. That is 100% a feminine spelling in the US.
If I were you, I would start going by
Ash now - and hide the fact that your name is
Ashlee. I'm not sure where in the US you are...but if you're in the midwest, the south, or a small town, you're in for it.