The Queen, 95, has "reluctantly" canceled a trip to Northern Ireland but is in "good spirits," said Buckingham Palace.
Prince Philip was previously awarded Consort of the Year by Oldie magazine in 2011.
Buckingham Palace had no comment or clarification on the queen's remarks.
The 95-year-old monarch has not held an engagement there since the start of the COVID-19 pandemic.
The Duke of Edinburgh died in April at the age of 99.
The former New York mayor imitated Queen Elizabeth in a 9/11 anniversary address described as "incoherent" by the "Late Night" comedian.
Sir Ken Olisa, the first Black Lord-Lieutenant for London, said race is a "hot conversation topic" in the royal household.
Some have said the move would be hypocritical, considering Prince Harry was stripped of his military titles earlier this year.
A spokesperson for Buckingham Palace responded: "We didn’t comment when this book was first released and won’t be commenting now."
The Duchess of Sussex received birthday wishes from Queen Elizabeth, Prince William, Kate Middleton and more.