Most of us want to keep our financial ducks in a row. Many of us work hard at it, living below our means, establishing emergency funds, saving and investing for retirement, and paying off or staying out of credit card debt. It's easy to think that it would be smart to pay off your mortgage early, and many financial experts recommend having it paid off by retirement. After all, the thought of having mortgage payments looming over you while you live on a more limited income can be unsettling. Give the decision some thought, though, because there are some reasons you may not want to rush to pay that loan off.