A lot of it comes fro negative press in the 80's and most of it was simply rumours. Yes - he did do the Royal Knockout which was a problem in the light that it drew on the royals as ordinary people , but more the press contempt as they weren't allowed to photograph due to a contract with a specific media group. He resigned his military commission which is still today seen as cowardly. Odd in a country where mostly people don't join the armed services themselves, but it was proof of his homosexuality for many. Then when Edward decided to follow the arts as his employment, further fodder. HE has for years been considered the black sheep of the family, the press often pit him against Charles or William as the villain in their soap opera. I have been told that we will never hear the truth of the Following William at St. Andrew's incident - but I know for certain that William threw a tantrum and event the staff at St. Andrew's wondered what the fuss was about. Whatever the truth was - didn't stop the tabloid from making up the story, I am told that the truth was actually a far better story, but William and Kate would not have come out clean there.
.