Eija

I always thought it was a cognate of the Icelandic name Eyja, which is a noun meaning "island". It was originally a shortening of feminine names that begin or end in "ey" (meaning "isle"), like Þórey, Eyrún etc. But maybe since Finnish is not a Nordic language, it is just a coincidence that they sound the same.

'Eijaa' is only used in old hymns, it comes directly from the Latin word 'eia', but it's not used in real Finnish. 'Heijaa' is also rare on its own and it comes from 'heia' which is an alternative on 'eia'.

---

Aija also originates from this name.

This is actually a diminutive for Erja or Merja or anything that turns into Eija in a child's mouth.

It would be interesting to know where you got this idea of "eijaa" as an exclamation. As far as I know, it's "heijaa". (Of course there can be regional variation in this but it just doesn't sound like a natural "happy exclamation" since "ei" means "no". On the other hand, "jaa" can mean "yes", so perhaps it's +/-.) Not that I have any other theory for the name's origin, but all the same.

I wonder if this name is related to the Latvian name Aija. It is my mother's name, and we believe it means something like the sound that water makes.