I spent a lot of the first decade of my professional life complaining about income taxes. But I never took time to understand how we are taxed, how much influence we have over our tax burden, and how friendly the tax code is to retirees, particularly early retirees.
My wife and I are taking a phased approach to early/semi-retirement. She cut back to working part time in 2012, five years prior to my retirement. She continues to work in that same role since I retired in 2017.
Making changes gradually gave us the courage to begin making major life transitions sooner than we otherwise would have. How does it impact our tax situation compared to full early retirement?
The challenge of comparison
It was initially my intention to share numbers from our actual tax returns over several periods: a household with two full-time incomes, one full-time and one part-time income, and then one part-time income only. The goal was to provide an apples-to-apples comparison of taxation in these different scenarios. It quickly became apparent how challenging that would be.
My last year of work was 2017. In 2018, the Tax Cuts and Jobs Act significantly changed the tax code. Our personal situation changed as well. My wife’s income has steadily increased each year. My writing provides some self-employed income. As such, I am now responsible for paying both the employer and employee portions of Social Security and Medicare taxes that I hadn’t as an employee.
We switched to a high-deductible health plan in 2017 and opened a Health Savings Account (HSA) that offers new tax advantages. Prior to that we’d never been eligible for a HSA.
We rented out our current residence during the second half of 2017 and first half of 2018, financed by a home equity loan with a variable interest rate. After breaking even in 2017, we sustained a substantial loss in 2018 when the interest rate increased. The loss decreased our taxable income that year.
Finally, we moved from Pennsylvania to Utah in July 2018. Our state income tax rate increased from 3.07% to 4.95%.
Apples to apples
It’s easier to demonstrate the tax implications of semi-retirement using a scenario based on our numbers while controlling key variables. This allows an apples-to-apples comparison to demonstrate key principles.
The assumptions about this couple are:
• They file taxes as married filing jointly with no dependents
• Each individual earns a full-time equivalent of $90,000/year
• Taxable investments create $12,000 per year of qualified dividends
• Taxable savings create $1,200 a year in interest
• While working, each spouse may contribute to a 401(k). Neither is over 50 so not eligible for a $6,500 catch-up contribution.
• Health insurance is provided through one employer
• The couple is not eligible to contribute to an HSA
• Taxes are based on the 2020 tax code
I double-checked tax calculations against the Federal Income Tax Calculator from the blog Go Curry Cracker!.
Federal taxes for a two-income household
In our first scenario, both spouses are working full-time. Total earned income is $180,000. Total taxable income with investments is $193,200.
The maximum contribution to a 401(k) account is $19,500 per person. Since both partners are working, the couple’s total contribution is $39,000, which lowers adjusted gross income (AGI) to $154,200.
Subtract the standard deduction of $24,800 to arrive at a taxable income of $129,400. Subtracting the $12,000 of qualified dividends gives $117,400 of income taxable at ordinary income tax rates.
The first $19,750 is taxed at 10% for $1,975. The next $60,500 is taxed at 12% for $7,260. The final $37,150 is taxed at 22% for $8,173. The total is $17,408.
Qualified dividends are taxed at 15% for married couples earning between $80,000 and $496,600. The dividends generate an additional $1,800 of taxes.
The total federal income tax paid by this couple is $19,208 on $193,200 of total income. Their effective tax rate is 9.9%.
The impact of cutting back work
The second scenario reflects the impact of one earner cutting back hours. After the birth of our daughter, my wife cut back to 30 hours per week.
Cutting back one spouse’s earnings by 25% reduces the household pretax income by 12.5%. But that income would have been taxed at the highest marginal tax rate, so what does this do to after-tax income?
The couple’s total earned income in this scenario is reduced to $157,500. Total income with investments is $170,700. Subtracting $39,000 for 401(k) contributions lowers AGI to $131,700.
Subtracting the standard deduction of $24,800 leaves $106,900 of taxable income. After subtracting the $12,000 in qualified dividends, it leaves $94,900 taxed as ordinary income.
The first $19,750 is again taxed at 10% for $1,975. Then next $60,500 is taxed at 12% for $7,260. Only $14,650 is now taxed at their marginal rate of 22% for $3,223. The total is $12,458.
The qualified dividends are again taxed at 15% for an additional $1,800 of taxes.
Their total federal income tax owed is $14,258 on $170,700. Their effective tax rate drops to 8.3%.
The impact of one spouse retiring
The third scenario reflects the impact of the full-time spouse retiring fully and the other spouse continuing to work 30 hours a week. This is what happened when I left my career in 2017. When married filing jointly, you can obtain a similar outcome if each partner worked 15 hours per week.
Total earned income in this scenario is $67,500. Adding in investment income brings it to $80,700. Since only one spouse is working, they can only contribute $19,500 to a 401(k), lowering AGI to $61,200.
Subtracting the standard deduction of $24,800 leaves $36,400 taxable income. After subtracting the $12,000 in qualified dividends, it leaves $24,400 taxed as ordinary income.
The first $19,750 is again taxed at 10% for $1,975. Only $4,650 is taxed at the lower marginal rate of 12% for $558. The total is $2,533.
Qualified dividends are taxed at a rate of 0% for couples married filing jointly if taxable income falls below $80,000. Thus no tax is owed on this $12,000 of investment income.
The total federal income tax in this semi-retired scenario is $2,533 on $80,700 of total income. The effective tax rate is 3.1%.
The tax code is much friendlier to low earners
Semi-retirement allows you to earn a substantial amount of income and pay little in federal income taxes. You keep about seven cents more of every dollar you earn when compared to working full-time, everything else being equal.
Semi-retirement also provides more time for your investments to grow. During this time, you eliminate tax drag on qualified dividends and long-term capital gains.
The 0% long-term capital gains rate provides the opportunity to harvest capital gains, meaning more of your future income could be tax-free with good planning.
Creating this apples-to-apples scenario is great for comparison. But life can sometimes be messier.
What about kids?
Kids don’t affect the calculations. They simply provide a $2,000-per-child tax credit.
Just multiply $2,000 by the number of children you have. Then subtract that number from the tax you calculated to get your final tax cost.
What about IRAs and Roth IRAs?
The ability to contribute to Roth IRA accounts and the ability to deduct traditional IRA accounts are dependent on your income in a given year. I couldn’t keep these variables constant for a fair apples-to-apples comparison.
Also, tax deductions are always more valuable when in the highest marginal tax rates. Roth IRAs are always more valuable for those with low enough income that they don’t pay federal income tax. In between, there is a gray area that is highly dependent on your personal situation.
What about health care?
Before I dove into the numbers on Affordable Care Act (ACA) subsidies, I thought the biggest drawback to semi-retirement was that having earned income would make health insurance unaffordable.
Our MAGI (modified adjusted gross income) last year was $73,000. Plugging that number into the Kaiser Family Foundation Health Insurance Marketplace Calculator with our specific geographic location and family demographics (two adults and one child) reveals that we would receive a subsidy of $725 per month and pay $595 a month for a silver plan through the ACA.
This is more than we pay to purchase insurance through my wife’s employer. But it is still reasonable.
Because we weren’t buying insurance through the ACA, we did nothing to optimize our MAGI. We could have contributed about $15,000 more to my wife’s 401(k) and contributed $12,000 to traditional rather than Roth IRAs.
Lowering our MAGI by $27,000, from $73,000 to $46,000, would drastically increase our subsidy and lower our premium costs. We would pay only $270 a month while receiving $1,050 per month in subsidies.
We could then meet our spending needs by selling off investments with long-term capital gains which would be taxed at 0%. However, the capital gains do count toward MAGI.
Chris Mamula retired from a career as a physical therapist at age 41. This is abridged from “The Amazing Tax Benefits of Semi-Retirement,” and you can read the full version on the blog “Can I Retire Yet?”