Queen Elizabeth Struggled Privately with the Divorces of Prince Charles, Prince Andrew and Princess Anne

Queen Elizabeth remained as stoic as ever as the divorces of three out of her four children played out publicly throughout the '90s. But behind the scenes was a different story.

In the upcoming biography, Queen of Our Times: The Life of Queen Elizabeth II, author Robert Hardman explores the Queen's private pain as the marriages of Prince Charles, Prince Andrew and Princess Anne each came to an end.

"Outwardly stoical, as ever, the Queen was finding the divorce talks deeply upsetting," Hardman writes in his book, which is excerpted in this week's issue of PEOPLE. "Another former member of the Household recalls that, every now and then, there would be a glimpse of her despair."

"It distressed her much more than she let on," a former staffer tells Hardman, recalling his attempt to put the broken royal marriages into some sort of perspective. "I said, 'Ma'am, it seems to be happening everywhere. This is almost common practice.' But she just said, 'Three out of four!' in sheer sadness and exasperation. One shouldn't underestimate the pain she's been through."

Despite what she was going through internally, the Queen never lost her cool during her horrible year, or "annus horribilis," in 1992. The year saw a fire destroy part of Windsor Castle and the collapse of Charles, Anne and Andrew's marriages — as well as the ongoing scandals surrounding Princess Diana and Prince Charles.

"I don't remember a single occasion when I went to see her and she exclaimed, 'No! What next?' " her former press secretary Charles Anson tells Hardman in Queen of Our Times, out April 5. "The issue was sometimes embarrassing, but she got on with it. It is immensely reassuring in those situations to work for someone who isn't knocked back."

Throughout, he adds, she was "never short; never irritable; completely steady."

Outwardly, the Queen chose "stillness" amid the drama surrounding Charles and Diana's split — an approach she learned from her father, King George VI.

"Her mother's strategy in these situations— to carry on as if they were not happening—had earned her the nickname 'imperial ostrich' among royal staff," Hardman writes. "The Queen's response, as ever, was to follow the example of her father, absorbed from his days at sea, and to treat adversity like the ocean."

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

"Storms will come and go, some worse than others," Sir John Major, who worked so closely with her through this period, tells Hardman. "But she will always put her head down and plough through them. The Queen has always lived by the doctrine, 'This too shall pass.' "

Hardman writes, "While the Queen has sometimes been accused of being slow to act, there has never been a charge of panic. Her default mode in the face of a crisis is stillness."