I have also seen same weird behavior specifically from altruistic stat when player has a choice to accept Saltire without an oath or make him swear it. Choice to believe him somehow tanked altruistic stat, if I remember correctly. And later, on choosing Saltire route (to accompany him), it somehow greatly lowered his affection stat.