Born in a log cabin, Millard Fillmore spent much of his youth clearing land and raising crops on the 130-acre farm that his father leased in New York’s Finger Lakes region. At age 14, his father, hoping to steer him away from farm life, apprenticed him to a cloth dresser. But he soon returned home after allegedly suffering severe mistreatment. Fillmore next apprenticed with the owners of a textile mill. Despite receiving only minimal formal schooling, he worked hard to educate himself and eventually became a schoolteacher. He also took an interest in the law, spurred on by his father’s landlord, a local judge. Following a couple of clerkships, Fillmore was admitted to the bar at age 23 and opened a practice near Buffalo, New York, where he made a good living and began to meet the area’s political leaders.

Fillmore was elected to the New York state legislature in 1828 on the Anti-Masonic ticket, which, as its name suggests, strongly opposed Freemasonry. He remained affiliated with that party, which at the time polled well in western New York, upon winning election to the U.S. House of Representatives four years later. But he soon cut ties with it to join the Whig Party, an amalgam of forces opposed to Democratic President Andrew Jackson. As a Whig, Fillmore served three terms in the House, lost a race for New York governor, became New York’s comptroller and then received a surprise nomination to be Zachary Taylor’s running mate in the 1848 presidential election.

With the Democrats split over the issue of slavery—some had left to form the anti-slavery Free Soil Party—Taylor and Fillmore took the White House with 47 percent of the popular vote. During Fillmore’s time as vice president, the Taylor administration largely left him out of the decision-making process. But on July 4, 1850, Taylor came down with a stomach bug after attending a Fourth of July celebration at the Washington Monument. His doctors, following the since-discredited medical practices of the era, gave him a mercury compound called calomel and induced bleeding and blisters. Within days, Taylor was dead and Fillmore had ascended to the presidency. The only other U.S. presidents never elected to the office were John Tyler (1841-1845), Andrew Johnson (1865-1869), Chester A. Arthur (1881-1885) and Gerald Ford (1974-1977).

Since the Constitution did not originally include a provision for replacing dead or departed vice presidents, the office has been vacant for about 38 of its 225 years. Fillmore, along with Tyler, Johnson and Arthur, had no second-in-command for the entirety of their terms. This situation is unlikely to repeat itself, however, as the 25th Amendment, ratified in 1967, allows the president to appoint a VP subject to the approval of the U.S. Congress.

Though personally opposed to slavery, Fillmore valued the preservation of the Union above all. As a result, he supported the so-called Compromise of 1850, a package of bills that allowed the newly formed territories of New Mexico and Utah to decide the slavery question for themselves; admitted California as a free state; banned the slave trade (but not slavery) in Washington, D.C.; settled a Texas boundary dispute; and authorized the use of federal officers to capture runaway slaves. “The long agony is over,” Fillmore wrote afterward. “[…] These several acts are not in all respects what I would have desired, yet, I am rejoiced at their passage, and trust they will restore harmony and peace to our distracted country.” However, the compromise did not hold, and the Civil War broke out in 1861.

Fillmore’s father purportedly owned only three books: a Bible, a hymnbook and an almanac. Yet Fillmore became a bibliophile anyway, carrying a dictionary with him at all times in order to improve his vocabulary. As president, he and his wife founded the first permanent White House library. He also reportedly raced to help fight a December 1851 blaze at the Library of Congress and then signed a bill to fund the replacement of all the books that had been destroyed.

Recognizing that the fugitive slave provision was the main concession to the South in the Compromise of 1850, Fillmore strictly enforced it. In so doing, however, he enraged northern abolitionists. At the 1852 Whig convention in Baltimore, Fillmore’s main support came from southern delegates, and he lost on the 53rd ballot to Winfield Scott, a veteran of numerous wars. In that November’s election, Scott carried only four of 31 states against Democrat Franklin Pierce, thus marking the beginning of the end for the Whig Party.

After leaving the White House, Fillmore returned to his law practice in Buffalo, but the lure of politics proved too strong to keep him away for long. In January 1855, Fillmore effectively announced his candidacy for president as a member of the anti-Catholic, anti-immigrant Know-Nothing Party, writing a letter in which he lambasted “the corrupting influence which the contest for the foreign vote is exciting upon our election.” He then took off on an extended trip overseas before returning to campaign on a platform of keeping the Union intact. The election did not go well for him, as he carried just one state, Maryland, and 22 percent of the popular vote.

During the Civil War, Fillmore supported the Union cause. But he expressed dismay at the emancipation of the slaves and the enlistment of black troops, did not follow most northern Whigs into the newly formed Republican Party and publicly opposed Abraham Lincoln’s re-election bid in 1864. Upon defeating the rebel army, Fillmore asserted that the North should extend to the Confederacy “every act of clemency and kindness in our power.” Unsurprisingly, he then backed Andrew Johnson’s more mild approach to Reconstruction rather than that of the so-called Radical Republicans.

Almost every time modern historians rank the U.S. presidents in order of greatness, Fillmore’s name appears near the bottom of the list. Even the White House’s official website calls him “uninspiring,” and in 1988 a Yale history professor quipped that “to discuss…Millard Fillmore is to overrate [him].”