In a low interest rates world, you should take a serious look at any loans you have. Although a 3.25% 30-year mortgage is often said to be "cheap money" or "almost free money" it’s neither cheap nor nearly free compared to the rate you earn on your bonds.
This chart shows the difference between the 10-year Treasury yield and the 30-year mortgage rate. Except the spike during the financial crisis, I would say the gap between the two right now is average to above-average. It’s not low. You are still paying a good premium on what you owe over what you earn.
Lending out money on one hand (holding bonds) and borrowing on the other at a higher rate doesn’t make sense unless you are betting on optionality: the mortgage rate is fixed and you are hoping to earn more eventually when rates go up. The problem with this line of thinking is that for every year you are holding out, you are paying a rate differential. By the time higher rates come, you are already way behind.
Contrary to popular belief, the interest rate on a 30-year fixed rate mortgage is NOT fixed for 30 years. It’s fixed until you sell the property OR 30 years, whichever comes first. Most people don’t stay in the same home for 30 years. When you sell and buy another home, you don’t get to keep your old rate. You get the market rate at that time. On average you get the fixed rate for more like 5-10 years. That’s not enough time for higher bond yields down the road to compensate for the deficit you accumulate in the early years. That is *if* we get higher bond yields down the road, which is not certain.
Also put yourself in the shoes of the investors behind your mortgage. Your mortgage eventually goes into a mortgage-backed security sold to investors. Those investors buy mortgage-backed securities for a reason. They have to bear the default risk. They have to pay a bank for servicing your loan. If they think they can earn a higher risk-adjusted return from buying the type of bonds you own through a bond mutual fund, they would do that and not bother with funding your mortgage.
Therefore you can’t expect to earn a higher risk-adjusted return from owning the same bonds they can buy. Maybe there’s a chance to make more from bonds institutional investors can’t buy (I Bonds, FDIC insured CDs, munis), but not from garden variety Treasuries, agency bonds, or investment-grade corporate bonds found in a total bond market fund.
Having realized that it’s more tax efficient to have bonds in taxable accounts and that they are really upside down in bonds versus their mortgage, the hypothetical couple in my "double the bond yield" initiative decided to sell $150k in bonds to pay down their 2.75% 15-year mortgage. In doing so, they saved extra 1% a year on $150,000, which is $1,500 or $750 per person.
Remember they had a $1 million portfolio, 40% invested in bonds. After paying down their mortgage, they still have $250k in fixed income including I Bonds they bought in a previous move.
This move pushed the "double the bond yield" movement to 37% complete. I will get there.
[Photo credit: Flickr user 401(K) 2012]