I do think Margaret's choice of vacation location played into the stories. It is easier to criticize someone vacationing in the Caribbean where it is warm and sunny with beaches than vacationing in the wilds of northern Scotland.
Going to the Carribbean meant travelling first-class on British Airways with her whole entourage and all its costs.
Taking an holiday in Scotland required nothing of that sort.