My name is Carey and I was always told it was derived from the Welsh Caer which means castle (Cearleon etc.) and as a personal name essentially means "from the castle." I was also told that it is used in Wales as the equivalent of Charles or Carl or the Anglo-Saxon name Ceorl. If all that is true it seems that it was originally a male name and only in relatively recent times became a female name too.
I was born in 1948 and christened Carey after my grandfather. He was christened Carey in 1878! He used to tell me that he was named after his parents' employer - a Lord Carey - they were "in service". Lord Carey's estate was in the Bristol area, (which is not far from Castle Cary). (Cary is pronounced just the same as Carey, even though most English people pronounce Cary like Carrie.) I think it is pertinent that Cary Grant (not his real name) came from Bristol. I like to think he chose his stage name because he knew of Castle Cary. One of my colleagues at work has a "choose a name for your baby" book. In it the name Carey is described as a Celtic name which means "he who lives in a castle".
I have never been able to corroborate the theory that the name Carey is of Irish origin. I have researched the ancestry of several Careys, and hav traced all of them to an Adam de Cari who was so named because he was the Lord of the Manor at a place that is now called Castle Cary, named so because the Castle was built at a village anciently named Cary or Cari (Celtic for rocky hill, according to the people at the Castle Carey museum). Several descendants of Adam have been prominent in British history, including the husband of Mary Bolyn.