I used to dislike
Oliver, but now I think it's oh-so-cute. I could see myself considering it one day.
I don't concern myself too much with popularity. I say if you truly love a name, popularity shouldn't matter. For example, I really love
Sophia, which is extremely popular here. I'd still consider it despite the popularity. But for you I guess it depends whether you value the name more or the fact that it's too popular for your taste.