I think it is pretty well know that he didn't get on well with Philip as a kid. And I think the queen was (a) awkward and shy and not very affectionate in any kind of physical way, and (b) very devoted ot her husband.
I think as a young woman she was wrapped up with Philip and perhaps wasn't ready for motherhood.. and was the kind of woman who was "content to leave childcare to nanny". There is a story - don't know how true - that when Diana was looking after the children because it was her nanny's day off, the queen said "I don't know Why Diana has to do this, there are plenty of housemaids."
I think she was a lot more "motherly" and affectionate to Andrew and Edward when they came along, but with Anne and Charles she was not very tactile. I don't know if she would ever be comfortable with expressing affection, esp physically.
I think Charles IS a very conscientious nervy sort of man, even now. He wants to do the right thing, agonises over what it should be, and as a little boy, he was problaby VERY nervous of PHilp and perhaps of his mother, who seemed distant and in his child mind, didn't protect him form Philip (or problaby the unpleasant Anne bullying him too).
I must look at the video properly another time, but yes I think he was a bit afraid of his parents, and felt more comfortable with his loving grandma, and she loved him because he reminded her of her shy seflf doubting husband.
I think in the present, he has grown closer to his parents, he understands them better and gets on with them, esp his mother...