Ashley isn't like Sidney/Sydney. Sidney at least has a fairly long history of use as a male name, so that even though it's been popular as a female name for generations, it could still be revived for males.
Ashley was never used much for males. Looking through the SSA's popularity lists, you won't find it very high up on the earliest lists, if you find it at all. When Ashley did become popular, it was almost exclusively used for females, and so it became, by default, a female name. Maybe not fair, but that's the way it worked out.