Aspen is synonymous with luxury — expensive vacation homes, upscale shopping, posh hotels and high-end art galleries — and, of course, skiing. But a large part of what makes the town so appealing as a destination is its sense of history. Impressive buildings from Aspen’s late-19th-century heyday give it real character, standing (mostly) firm against the pressures of sky-high real estate prices. Fascinated by the area’s dramatic swings in fortune, we joined the Aspen Historical Society for a private two-hour walking tour to learn more about the town’s past.

Starting at the Wheeler Opera House, built in 1889 by Jerome B. Wheeler, a co-owner of the original Macy’s department store, our guide set the scene: By the 1890s, Aspen had become one of Colorado’s largest cities, producing one-sixth of the nation’s silver. But just as suddenly as the boom happened, it went bust: In 1893, the silver market plummeted and so did Aspen’s population, setting off a 50-year slump known as the “Quiet Years.”

The recipe for Aspen’s revival came down to three unlikely ingredients: a small club of ski enthusiasts, the Army’s 10th Mountain Division training nearby and a three-week music festival brought to the struggling town in 1949 by visionaries Elizabeth and Walter Paepcke. By the early 1950s, Billie Holiday and Duke Ellington were playing the Red Onion nightclub, and Aspen had hosted its first international downhill ski race.