somewhere down the line they've lost the whole "plot" aspect and the atmosphere...

I couldn't agree more.

I still haven't played Koudelka, so I can't comment on it, but Shadow Hearts was head and shoulders above the others in terms of plot and atmosphere.

I blame the lack in part on the fun side quests you mention/. I felt they so watered down any sense of direction and purpose as to render the game almost without meaning.