The rumours with David and Wallis are interesting, to say the least.
With David, the speculation comes up because despite being somewhat of a playboy he never had children. The reasoning here is often assumed to be infertility because of the mumps he had growing up, although I've also read suggestions that his sexual behaviour wasn't one that resulted in baby making, or that he did have children but they were covered up and went unacknowledged.
With Wallis, the fertility speculation is that despite having been married three times and having had lovers, she never had children - although, again, it is alleged that she had a love child with David who was put up for adoption and went unacknowledged. Here the reasoning varies from her being naturally infertile, her being intersex or a man, her having had either her tubes tied or a hysterectomy without any of her husbands having known, or the result of an abortion gone wrong.
I'm not certain about the Yorks' fertility problems, although if I remember correctly the Queen Mum had problems with her deliveries and possibly her pregnancies as well, so I think any problems there may have been more on her end. That said, as they had two children we can assume that neither of them was infertile although they may have had difficulties conceiving (or may have simply intentionally delayed conceiving).