'So you're planning to write a book about the Queen Mother,' said the Duke, exchanging a conspiratorial smile with his wife.
'Well, we shall have to be extremely careful what we say on that subject, won't we darling?'.
'Why is that, Sir?' I inquired innocently, although I was well aware of the reason.
The Duke, only months away from being diagnosed with inoperable throat cancer, was interrupted by a convulsive spasm of coughing.
He cleared his throat and added: 'I hope your book will tell the truth, instead of all that gush they dish out about her.
'Behind that great abundance of charm is a shrewd, scheming and extremely ruthless woman.'
He must have noticed my surprised reaction, for he quickly added, with his most charming smile: 'But, of course, you cannot quote that.'
The Duchess was less inhibited. 'The Duke would have loved to return to live in the land of his birth,' she said. 'But our way was blocked at every turn. We were never allowed to go back, and we never will be allowed. Not until the day we die. She will never permit it. When we are dead, perhaps she may at last forgive us'.
When I asked her the reason, the Duchess's right arm shot out as if she was taking aim with a gun and she said: 'Jealousy.'
'Jealousy of the Duke?' I wondered. 'No!' cried the Duchess, and for the first time her southern American origins were audible. 'Jealousy of me for having married him.'
The Duke, who appeared vaguely uncomfortable with this topic, murmured: 'Well, it's hard to explain. But, yes, Elizabeth (the Queen Mother) was rather fonder of me than she ought to have been. And after I married Wallis, her attitude towards me changed.
'My sister-in-law is an arch-intriguer, and she has dedicated herself to making life hell for both of us.'
The Duchess interjected: ' Somebody once told me that even Hitler was afraid of her.'
The exiled Duke and Duchess then proceeded to make a series of astonishing allegations against the Queen Mother. All of them were dramatically at odds with her image as the nation's favourite grandmother.
They were equally at odds with the effulgently glowing, reverent, completely sanitised and at times absurdly hagiographic contents of William Shawcross's official biography launched this week.
The Duke of Windsor insisted he possessed 'proof positive' that the Queen Mother had personally engineered the Duchess's exclusion from royal rank - an action now judged constitutionally illegal and known to historians as 'the Depriving Act'.
'It was her doing entirely,' said the Duke. 'It was not something my brother, the King, would ever have done, left to himself. But he deferred to her influence, just as her daughter does now.'
On the outbreak of war in 1939, the Windsors' enforced return to Britain was met with complete ostracism by the Royal Family, led by the Queen Mother's announcement that she would not meet the Duchess.
'What are we going to do about Mrs S?' she wrote to her mother-inlaw, Queen Mary, in a less than kind reference to the woman who had been her sister-in-law for more than two years.
'I personally do not wish to receive her,' she added implacably - making it clear the initiative was hers, and hers alone.
After the fall of France in 1940, when the Windsors fled Paris, they were briefly marooned in Lisbon, where it was reported to the Germans that 'the Duke and Duchess have less fear of the King, who was a complete nincompoop, than of the shrewd Queen, who was intriguing skilfully against the Duke and particularly against the Duchess'.
Referring to her husband's appointment as Governor of the Bahamas, the Duchess told me at our 1971 meeting: 'We were then offered this pathetic little job in a ghastly backwater. It was designed to get us out of the way, but she even tried to stop that.'
Indeed, the Queen wrote to the Colonial Secretary, Lord Lloyd, predicting that if the Duke became Governor, 'a very difficult situation will arise over his wife'.
The fact that the Duchess 'has three husbands alive (including her two exes - a U.S. navy pilot and a shipping executive), will not be pleasing to the good people of the Islands'. Britons were used to 'looking up to the King's representatives'.
Then she added a sentence that puts her utter loathing of Wallis Simpson beyond doubt: 'The Duchess of Windsor is looked upon as the lowest of the low.'
When the Windsors, despite the Queen's objections, arrived in Nassau, they were confronted by Foreign Office memos instructing the local ladies not to curtsy to the Duchess.
'That was her doing, of course,' said the Duke in 1971. 'Wherever we looked, we could see her hand against us.'
The Windsors' most astonishing claim concerned their visit to the United States in 1941.
The Queen Mother, they alleged, enlisted the help of Special Branch in London; of her brother, David (later Sir David) Bowes Lyon, who was posted to the British Embassy in Washington; of the British Ambassador, Lord Halifax; and of her friends, President Franklin D. Roosevelt and his wife Eleanor, to have them followed and spied on 24 hours a day by FBI agents.