All other savings, including your emergency savings, are depleted. There’s a pile of money in your 401(k). It’s a tempting solution to your money problems, but do you really want to mess with your retirement nest egg?

And how, exactly, is that going to affect your retirement, especially in your later years?

While financial planners certainly don’t promote raiding your 401(k), most say that sometimes it can’t be avoided. Their advice: If you do have to go that route, make sure you have exhausted all other avenues first. And if you are disciplined enough to repay the loan, they see nothing wrong with it.

“I don’t believe for anyone or any circumstances you should say never,” says Anthony Saccaro, founder and president of Providence Financial and Insurance Services in Woodland Hills, Calif. “Every circumstance is different. There are times it is advisable and times that it is not.”

But, he adds, “More often than not, I am trying to look at other ways other than 401(k),” he says.

Most 401(k) plans will let you borrow up to $50,000 of your vested balance. The loan must be repaid within five years, unless the loan is used to buy the participant’s main home. Many times, the interest rate on a 401(k) loan is quite low, typically a percentage point or two above the prime rate, now 3.25 percent.

If you leave your job, you’ll have to repay the loan immediately, or pay taxes on the remaining balance. If you’re younger than 59½, that means a 10% penalty tax on the remaining balance, plus ordinary income taxes.

But taxes aren’t the only reason to be wary of a 401(k) loan, says Jeanne Thompson, vice president at Fidelity Investments. A Fidelity survey of 12 million Americans, focusing on trends and pitfalls in tapping 401(k) loans, raises some concerns.

“What we found most interesting is there is a population of people that tend to be serial borrowers,” says Thompson.” They don’t just take one loan, but they take multiple loans. “A 401(k) is mean for retirement,” Thompson says. “If you continue to take loan after loan, it can really impact your retirement.”

Stewart says it can also be a bad idea in a bull market. “If the market were to continue to rally, it could result in a lost opportunity,” he says.

Sometimes, a 401(k) loan is the only alternative, and you can’t dismiss it entirely. “It’s silly for anyone to say it’s always a bad idea to borrow money from your 401(k),” says Seth Stewart, president and founder of Plan My Benefit in Nashville.

“It comes down to discipline,” says Stewart. “If you are a very disciplined individual and stick with terms and conditions of the loan, it is fine.”

Stewart says he has small-business clients who have had to borrow from their retirement savings, and he sees nothing wrong with that.

All the planners say if you start an emergency fund you can tap that before looking to your retirement savings. But if you do borrow from yourself, how do you get back on track?

“Keep current on payment,” Nordquist says. “If you find an opportunity to prepay it, do it. If you get a pay increase or get taxes back, always look for opportunity where can you pay it back sooner.”

Thompson also recommends using the catch-up provision, which allows people over 50 to contribute an additional $5,000 a year to their accounts.

© 2013 USA TODAY. All rights reserved.