3 comfortable retirees all with around $2M saved. What did they all have in common? All own homes.

For many Americans, retirement advice is limited to encouragement to save more or warnings that they haven’t saved enough.

But most people get little guidance or give little thought to what to do with all those savings once they reach this next chapter. Whether they are decades, years, or months from retirement, it can be hard to imagine the life that 401(k) ultimately buys. And it is incredibly difficult to erase the anxiety about whether you are spending too much each year.

“People haven’t really thought through what retirement is going to look and feel like,” said Robert Laura, a retirement coach in Brighton, Mich. “They quickly realize they have a lot of time to fill. You need more than a financial plan,” he said.

Comfort and anxiety were tested in the past year even for those with significant savings. Rising inflation dented nest eggs more than many had budgeted or expected. Meanwhile, a declining stock market cut into many retirees’ savings.

We spoke in depth with four retirees who saved enough to build comfortable retirements, with net worths ranging from roughly $2 million to $4 million. They shared insights about how they spend their time and money, what has given them joy or anxiety, and how their expectations of life in retirement measured up to the reality. The four live in different parts of the country and are at different phases of life—with one just entering retirement at 61, while another is in his 80s.

Among other things, they offer insight into some of the challenges that retirees—even those with substantial savings—face today. They also shared advice on the importance of having a sense of purpose in your post-career years, keeping engaged through paid or volunteer work, and periodically re-evaluating one’s needs and desires in response to aging and other changes.

Here are their stories:

John Fitzgerald

Savings and Investments: $2 million

Annual Spending: $144,000

When John Fitzgerald retired as a police lieutenant about three years ago after 33 years on the force, his deferred compensation plan was worth about $1.7 million.

These days, due to the stock market pullback, it is worth about $1.3 million. The 61-year-old Brookeville, Md., husband and father of three is concerned as he’s counting on that money to help fund his lifestyle.

“I see my hard-earned money slipping away every day,” he said.

Mr. Fitzgerald considers himself fortunate as he also has a roughly $6,900 monthly pension after taxes and insurance. He has about another $350,000 saved in other accounts including bank accounts and a college savings fund.

So far, he and his wife, Jill Fitzgerald, 58, haven’t made any changes to their portfolios. Mrs. Fitzgerald works as a writer and editor and has about $400,000 saved for retirement.

The couple is feeling the effects of inflation. They help pay the college tuition for their youngest son, a bill that they expect to rise roughly 5% to about $35,000 this fall.

In total, Mr. Fitzgerald estimates they pay about $12,000 in monthly expenses including mortgage payments for their Maryland home and a Delaware beach house. The couple has about $400,000 in debt including mortgages and car loans.

They have cut the amount of groceries they buy, but their bill jumped to about $600 a month from about $300 a month, he said. They have eliminated favorites such as bagged salads as the roughly $4.59 per bag price isn’t worth the recent $2 price increase, Mr. Fitzgerald said.

Higher prices aren’t stopping Mr. Fitzgerald’s extensive volunteer schedule, however. From March through November, he coaches a baseball team for 11-year-olds and is commissioner of a local youth baseball league. During the season, he spends about 30 hours a week on this work, which frequently occupies most of his weekends.

“My wife allows it because she knows I love it,” he said.

When it isn’t baseball season, the days can sometimes drag, he said. He recommends other retirees find a passion to look forward to so they don’t get bored. Looking ahead, Mr. Fitzgerald would like to sell his Maryland home in about four years and move to Florida to save on taxes.

James Compton

Savings and Investments: $1.5 million

Annual Spending: $100,000

James Compton, 84, only recently fully retired.

While he left his job as president and chief executive officer of the Chicago Urban League more than 15 years ago, he has kept himself busy by serving on corporate and nonprofit boards.

He was still working full time when he joined the boards of Ariel Investments, Commonwealth Edison Co. and the Field Museum of Natural History. Thanks to some of those board seats and others, he earned about $150,000 a year, which helped replace a large portion of his full-time work salary. Over roughly the past decade, he said he has aged off all of the boards.

His social pace is slower these days, but he still keeps active. Mr. Compton used to spend about $125 a week on lunches out with contacts and people who sought career advice. These days, due in part to rising prices and because one of his favorite lunch spots closed, he spends about $60 and goes out about twice a week instead of five times.

He still enjoys giving to several of his favorite charities, including Morehouse College, but is reducing the amount because of inflation and the impact of the stock market pullback on his portfolio. He estimates he’ll donate about $8,000 in total this year to various charities instead of the roughly $15,000 he gave last year.

Mr. Compton valued his portfolio at around $1.5 million, 70% of which is invested in stock mutual funds. He is concerned about market volatility and the general climate of uncertainty right now.

“I should probably take less risk, but I still sleep well at night,” he said.

As interest rates have risen, he’s concerned about the $200,000 adjustable-rate mortgage he’s still paying off on his three-bedroom Chicago townhome. He is unsure what the loan’s rate will be when it resets next year so he is considering his options including eventually selling his home.

“Don’t go into retirement with a mortgage,” he said.

Social Security checks help him pay for the townhome’s upkeep, which he estimates to be about $2,000 a year.

He devotes more time to physical fitness now than he could when he was working. Four days a week, he’ll head to a local health club to lift weights or take a cardio class. “My doctor tells me I’m in perfect health so I plan to keep learning and connecting with others for as long as I can,” he said.

Judy Hall

Savings and Investments: $1.8 million

Annual Spending: $110,000

When Judy Hall retired in 2005 at 58, she had about $2 million and no clue how she was going to fill her days. To gather ideas, the former human resources executive at Berkshire Hathaway Inc.’s General Re took a seven-week road trip to visit retired friends.

“I wanted to know what retired people do all day,” said Ms. Hall, 75, who describes herself as a former workaholic.

When she returned to her Manhattan apartment, Ms. Hall increased her volunteer work at St. Bartholomew’s Church. She organized a conference and filled in for an employee on leave. She also became the church’s liaison to a homeless shelter.

“They called me the uber volunteer. I loved it,” said Ms. Hall, who discovered a new sense of purpose. “My mantra is to wake up in the morning and make a difference in someone’s life.”

At the shelter, Ms. Hall met a family she remains close with.

Five years ago, Ms. Hall sold her one-bedroom apartment and bought a $450,000 condo in Naples, Fla.

“I need to figure out what my next act is. I’m not a person who can sit around,” said Ms. Hall, who tutors in an after-school program and serves on the board of trustees of her alma mater, Roanoke College in Virginia.

For the past 30 years, Ms. Hall has owned a beach house she named Camp Cocktail in Ocean City, N.J. She frequently entertains guests, including a group of high school friends who printed “Camp Cocktail” T-shirts.

One mainstay in Ms. Hall’s life is her 10-year-old goddaughter, who is a member of the family she befriended at the shelter.

Ms. Hall said it was hard to get used to living without the structure of a job and regular paycheck.

Throughout her 37 years at Gen Re, Ms. Hall said she saved 6% a year in her 401(k), where she received a 6% company match. She amassed $1 million by retirement, and swapped her pension for a lump-sum that added another $1 million to the balance.

Although Ms. Hall retired shortly before the 2008 financial crisis, her long-term investment returns have been high enough to replace most of the money she has withdrawn from her savings, leaving her with $1.8 million of the $2 million she retired with.

Ms. Hall spends more these days on food, gas for her Honda Accord, and airfare to Roanoke. But when she left New York, her spending declined to about $110,000 a year from $200,000. She receives $25,000 in Social Security and donates about $30,000 a year.

With two homes, Ms. Hall rarely travels, but she hopes to visit Ireland and Australia next year. Otherwise, she said, she’s more interested in downsizing than spending. “I don’t want more stuff,” said Ms. Hall.

Content courtesy of WSJ, Veronica Dagher and Anne Tergesen