Most Japanese names sites list "Kaori" as a masculine name, meaning "add a man's strength".
However, I did find one site which lists it as a feminine name meaning "fragrant":
http://www.angelfire.com/anime/darkreflection/names.html
The difference between the masculine and feminine versions of "Kaori" most likely depend on how the kanji characters are written.
An older book of Japanese names which I have shows "Kaoru" to be a feminine name meaning "fragrant".
-- Nanaea