This name is indeed Norman, but you have the wrong derivation. It comes from giffre + roi = the eaglehawk of the king = the king's trusted messenger --- back in the days we were labeled by job or location (cooper, river, etc.)
I have found the earliest use of "Geof", was the peagan word "Geofu". Which meams "Gift". Turned into the name in early Germania. As Christianity emerged it was not acceptable by Romans to consider any other child as a gift other than Christ. It was then changed to Geoffrey, meaning "Gifted one" eventually it's meaning lost, and spelling and pronuciation changed as Christianity became more prominant. This explaination was provided to my mother in Germany before I was born and was the reason why she chose this name for me.
Following the "G" line may indeed lead to Germanic roots, yet when following the "J" line, the history of the name leads back to the eldest son of Noah, Japhet. If you can manage to pronounce these letters as: jeffy, then you can see the relationship. The name becomes a feminine name as 'Yaffa', and a last name in 'Japhee'. The name make further travels into Espanol where 'jeffe' means Boss, Note the similarity of the homophone 'chief' to 'chef' to 'jeffe' Additionally: chaperone, schaeffer, gaffer, and jeopardy. The last being the greater clue in the mystery of names.