I always thought Sam was simply repressed, mainly due to the trauma of witnessing his father attack a woman when he was a child of four; and also due to being deserted by his father at the same age. Sam pushes people away because he is afraid of being abandoned.

(Way-back-when I wrote a timey-wimey story about what Sam might have been like without that particular trauma: Five little words. I put some hints in the story about how I thought alt!Sam's life with Maya would play out if you're interested.)