According to the White House Historical Society and Dolley’s personal letters, President James Madison had left the White House on August 22 to meet with his generals on the battlefield, just as British troops threatened to enter the capitol. Before leaving, he asked his wife Dolley if she had the “courage or firmness” to wait for his intended return the next day. He asked her to gather important state papers and be prepared to abandon the White House at any moment.
The next day, Dolley and a few servants scanned the horizon with spyglasses waiting for either Madison or the British army to show up. As British troops gathered in the distance, Dolley decided to abandon the couple’s personal belongings and instead gave the order to save a full-length portrait of former president George Washington from desecration. Dolley wrote to her sister on the night of August 23 of the difficulty involved in saving the painting. Since the portrait was screwed to the wall, she ordered the frame to be broken and the canvas pulled out and rolled up. Two unidentified “gentlemen from New York” hustled it away for safe-keeping. (Unbeknownst to Dolley the portrait was actually a copy of Gilbert Stuart’s original).
Although President Madison and his wife were able to return to Washington only three days later when British troops had moved on, they never again lived in the White House. Madison served the rest of his term residing at the city’s Octagon House. It was not until 1817 that newly elected president James Monroe moved back into the reconstructed building.