Yes, but that doesn't change the fact that Caroline and Carly mean "free man" and that it is sort of odd to hate Madison just because it means "son of...".
The way I see it it’s kind of like how people used to use “man” to refer to mankind to humanity in general. A free person. Or like the meaning of Alexander, defender of mankind. Not literally talking about a man. But as far as I’m aware Charles just means “man”. Or maybe that’s just me.