To be honest here, I don't think that infidelity was the main issue of Diana's paranoia with Camilla. When Diana and Charles first started really "courting", Diana realized that Camilla and Andrew Parker-Bowles were part and parcel of Charles' close circle of friends and everyone knew of Charles and Camilla's past dalliances. Her sister Sarah, I believe, dated Charles after Camilla married Andrew. Diana had to be aware of this.
What I think was really Diana's main concern and resulted in her paranoia about Camilla was the intimacy which Charles and Camilla had. The intimacy of friendship and companionship that C&C shared over love of common interest, the same circle of friends and a bonding of understanding each other. This is what I think Diana wished she could have formed with Charles.. the intimacy. And in a fairytale storybook way, perhaps she imagined that once married, he would direct all that onto her.
Who actually slept with someone else first is besides the point. The two of them never really bonded in a close friendship nor did the efforts made during the marriage take root. That's why the attacks in public format came about.
The Panorama interview was the proverbial "straw that broke the camel's back" so to speak. What is sad is that its been said that at the time of Diana's death, her and Charles actually were on the verge of a good, friendly relationship.