As mainstream as
Olivia is, I still find it matronly.
I love olive trees and what they symbolise is lovely too.
Olive, is simple, sweet and funky and I like it very much.
While it's fine to only find it food like (I can understand that) I don't see how olive being a noun (colour, tree and food item) prevents it from being a "name". It's been used as a name for a long time. I haven't heard anything quite so ridiculous I'm sorry. Using that logic would mean many long used noun names would be non names too.