I think both Lydia and Hazel are gorgeous names and they don't scream trendy to me either. The only thing that comes to mind with either name is that they sound lovely and classic to me. I wouldn't worry about their current ranking on the charts, it's not like they are #1 and #2 or something and doubt either would get to that level of popularity. Both sound lovely with older brother August too, so you can't go wrong imo.
Sarah -
Wife to DH Paul (Since July 4th 2009)
Mama to Jasper Emmanuel (Since August 6th 2010)
Due with baby #2 (and boy #2!) in August 2014