According to The Guardian:
[T]he Queen paid a rare visit to Downing Street for a lunchtime gathering of all but one of her surviving prime ministers. The Queen and Duke of Edinburgh were joined by four of the twelve prime ministers to have served during her 60-year-reign.
Winston Churchill, who was much taken with the 25-year-old sovereign and nurtured her through her early years, extended the first dinner invitation to Downing Street on the occasion of his retirement in 1955. Equally charmed was Harold Wilson, who invited her at the time of his resignation in 1976.