Passive Income to fund (an early) Retirement

You’ve saved and built your nest egg over many years. A lot of it is probably in your tax-sheltered retirement accounts. How can you retire “early”, because there are penalties if you access your tax-sheltered retirement funds (401ks, IRAs) too soon? For example, the general rule is that you will be penalized 10% if you withdraw tax-sheltered funds prior to reaching age 59 1/2. What if you don’t want to wait until then?

There are a few exceptions or other ways to access tax-sheltered accounts before age 59 1/2, but another approach is to build “passive” income to replace your needs until you can access the tax-sheltered money. If I could go back in time, I would have started building our passive income sooner. We just started working on this in the past few years.

We are estimating a budget based on our expenses after we stop working, but before we can tap Social Security and tax-sheltered accounts. Keep in mind, much of our budget is dedicated to saving for retirement, so this substantial part of our “expenses” will go away, remember to “pay yourself first!” Ideally, we will time it to minimize or eliminate other big ticket items: mortgage, tuition, etc. Our youngest child will still be in school during our targeted retirement, but we plan to payoff our mortgage. Since I love cars, I will probably still have a car payment.

Dividend Income

We have a brokerage account with TD Ameritrade that we opened around 2000, but we did not contribute to it regularly until the past couple of years. In this account we focus on Dividend Growth Investing (DGI) and also Income Based Investing, while also trying to be tax efficient. Because this is a taxable account, we select tax advantaged investments, including ones that pay Qualified Dividends (QDI) or even tax-exempt divvies.

We make regular (monthly) contributions to this account as part of our budget. We are targeting a rate of 8+%, so $100K would generate $8k year without touching the principal. This return may seem high, but I mix my own research with the High Dividend Opportunity service on the Seeking Alpha platform. With COVID-19, murder hornets, and other debacles of 2020, it has worked out ok so far, but we’ll see how it tracks and continue to adjust as needed. This platform has provided exposure to several other types of investments that I had not used in the past. In addition to existing positions in common stock, MLPs, and mutual funds, we now have the following as well:

• Closed End Funds (CEF)
• Preferred Stock
• Exchange Traded Funds (ETF)
• Bonds
• Baby Bonds

For us, the key to these investments is that they pay dividends, interest, or other distributions. The tax treatment varies. Some examples below:

Ticker	Investment Name	Investment Type	Tax Treatment	* Yield
T	AT&T	Common Stock	Qualified	7.08%
EVN	Eaton Vance Municipal Income Trust	CEF	Tax Exempt	4.33%
CDR-C	Cedar Realty Trust, 6.50% Series C	Preferred Stock	Income	7.36%
GECCN	Great Elm Capital Corp., 6.50% Notes	Baby Bond	Income	6.66%

Focusing on dividends and distributions has been a change from my early investment days. In the beginning, I would to “buy low” and “sell high”. Makes sense! With this approach, you purchase a stock, like Amazon (AMZN), when it’s “low”, trading at $250/share. You sell it later when it’s $2,500 a share, you made 10 times your original investment (a 10 “bagger”). Obviously, this works best when stocks appreciate a lot and the growth is called “capital gains“. You pay tax when you sell the stock and it’s classified as either short term capital gains (holding for one year or less) or long term capital gains (holding over one year). Generally, short term capital gains are taxed as “income”, but long term capital gains have a more favorable structure.

A dividend stock generally makes regular payments (usually quarterly, but could be monthly or semi-annually) to the party that holds the stock on particular date, one day before the “ex-dividend” date. For example, T (AT&T) pays $2.08 per share annually ($.52/quarter). I have 500 shares of T, so in a year I would get paid $1,040. I don’t have to do anything, just hold the stock. This is why it’s called passive income! You can see how this could be beneficial if you need income but you’re no longer earning a paycheck from “work”.

I can still sell shares for capital gains, but if T continues to pay dividends, I may not care too much whether the stock appreciates or not. In fact, if I reinvest dividends, which I do in my tax-sheltered accounts, I don’t mind if it trades “sideways” so I can get more and more shares and increase my eventual income stream. In fact, if one of my holdings has a large capital gain, it can make my “Yield on Cost” (YOC) look great, but it may make sense to reallocate the capital gains and increase my overall income. It took me awhile to grasp this, even though I had heard the principle. It helps to write down an example to understand, such as this article I wrote on YOC.

Some companies regularly increase their dividend every year. For example, the “Dividend Champions” is a listing of stocks have have increased their dividends every year for 25+ years, including T! Buying stock in companies like this is often associated with the Dividend Growth Investing (DGI) approach. There is an even more exclusive club of “Dividend Kings” that have increased their dividends every year for 50+ years!

For our AT&T example, the tax treatment for dividends is “Qualified“. This is similar to the distinction between short term capital gains and long term capital gains. You pay a more favorable rate for qualified dividends vs non-qualified. Non-qualified are generally taxed the same as ordinary income. Qualified dividends are taxed at a lower rate with a structure similar to long term capital gains. It may make sense to hold these type of investments in a non tax-sheltered account, but you need to consider your own situation and can consult your accountant or financial planner for guidance!

If we look at the EVN (Eaton Vance Municipal Income Trust) example, the dividends are actually tax exempt! It pays a lower yield of “only” 4.33%, but since it’s not taxed, this is the equivalent to 6.46% yield based on our income! Not bad! I used this Tax Equivalent Yield calculator at BankRate.com to find the equivalent yield, so many great tools available online!

Since EVN consists of municipal holdings, it is fairly consistent with a 5Y beta of .08 per Yahoo finance. Beta is a measure of a stocks volatility or how much the price fluctuates, compared to the market. An index like the S&P 500 would have a beta of 1.0, since it is a good measure of the overall market. A beta > 1.0 means it is more volatile than the market, so .08 means it is not very volatile. This may not be a good equity to buy for growth or capital appreciation, but it fits well in my portfolio now from a “Capital Preservation” perspective. Look at the chart of EVN below for January 1999 to February 2021 from Schwab.com. It looks somewhat volatile at first, but then note that the amounts on the y-axis are fairly narrow and over this long period of time, it has mostly traded in a range between $11 and $13!

When you’re younger, you have more time to “make up” for mistakes. You can generally be more aggressive. A stock with a higher beta means more volatility, which could be worse losses than the market, but more optimistically it could be higher gains. We have seen the huge growth in tech stocks like TSLA over the past couple of years. We have seen the insane volatility with Game Stop (GME) and other heavily shorted Reddit stocks. According to Yahoo Finance, the 5Y beta for TSLA is 2.09, lots of volatile growth! The same metric for GME is -1.95, ouch!

We will look more at different metrics and ways to compare stocks, and also different mechanics for other investment types in other articles. A lot of it may appeal to nerds 🤓 or those who like money. I do notice people’s eyes glazing over when I start rambling about some of the technical details…

Below is the max chart for TSLA for 5/25/2010 to 2/26/2021 from Schwab.com. Look at the Y-Axis! This reflects a 5:1 split at the end of August 2020. I did have TSLA, but sold my last lot in January 2020 for a 435% gain, which sounds good until you see what happened after… 🙁 There is definitely some “Fear of Missing Out” FOMO, but we still hold other tech stocks that have been big winners. In many cases, we are divesting them over time to sharpen our focus on capital preservation and income. It’s hard to part with all of them completely, and we will generally keep a small position “just in case”.

For fun, here is a chart of Game Stop from 2/11/2002 to February 2021. Yikes! Definitely potential to get rich or lose lots of money. Don’t forget:

“the markets can remain irrational longer than you can remain solvent.”

John Maynard Keynes

Real Estate

I have often considered buying a rental property or a vacation home as an investment. I don’t think we have the temperament or frankly, the luck, to pull this off. I read about real estate crowdfunding online and signed up on the Realty Shares platform a couple of years ago, and it has been a mixed bag.

I had 6 debt investments on Realty Shares, 4 exited successfully, with a return of approximately 8%. The remaining investments are still pending, having run into problems even before the COVID-19 debacle of 2020. I received about 50% of the principal on one $5k investment. There is a forbearance agreement that expired, so we’ll see how much, if any, I get of the rest. The other one stopped making payments, but has since resumed and I’m cautiously optimistic that the sponsor will be able to refinance and close the deal successfully. This was a wake-up call and taught me to be much more cautious going in, the initial successes made me overconfident. In retrospect, these were “fishy” deals, including a 2nd lien loan. What was I thinking!? A phrase I heard a lot when I first started researching different investments and it is still relevant as ever: do your own due diligence!

Incidentally, you have to be an “Accredited Investor” to participate in the Realty Shares platform. As defined in Rule 501 of Regulation D of the SEC, a few of the ways for individuals to qualify include:

• A net worth of $1,000,000 or more, excluding equity in the primary residence
• An individual income of $200,000/year or joint income of $300,000/year for the last two years with the expectation of the same for the current year.

A few years ago we reached the $1M milestone excluding our primary residence, so here was a benefit that we could get out it! More recently we hit $2.3M net worth, which supposedly means we’re rich, but we know we have not achieved our goals yet.

I signed up for a similar platform called Realty Mogul that also required accredited investor qualifications. For Realty Mogul, I had an interview and shared some paperwork as backup. With Realty Shares I just had to fill out a form online, which seems a little sketchy in retrospect. Hopefully this is a sign that Realty Mogul is the real deal, but so far been to scarred to pull the trigger on any deals. Keeping an eye on how things shake out with COVID-19 and the economy before giving crowdfunding another try.

We switched to the Fundrise platform since the fall of 2019. I have been happy with them and their improved quality, transparency, and communication. Fundrise outperformed the markets and REITs over the first half of 2020, with a return of roughly 5%. This matches what I have seen in my own account, but generally the return has been closer to 10% over the past several years. So far, it’s nice to have some exposure to real estate for more diversification, without the pain of managing a physical property. We also contribute to Fundrise on a monthly basis as we try to build our passive income.

Where are we now?

Source	Account Type	Annual Amount
Roth IRA	Tax-Sheltered	$10,000
Roth IRA	Tax-Sheltered	$10,000
401K	Tax-Sheltered	$20,000
401K	Tax-Sheltered	$20,000
401K	Tax-Sheltered	$2,000
Brokerage	Non Tax-Sheletered	$7,000
Fundrise	Non Tax-Sheletered	$1,000
		$70,000

Our passive income across all accounts overall is about $70K/year, but out non tax-sheltered passive income is only generating about $7K/year. This is growing as we continue to contribute to it, but we need to bump this up significantly! Our target retirement adjusted budget will be less because we won’t have the following items:

• Saving for Retirement: $7,250/month
• Mortgage: $3,000/month

However, we are still short of the goal if we wanted to rely on Non Tax-Sheltered income. Achieving the target monthly budget post-employment in taxable accounts is a daunting task. We have some other ideas: Maybe I can start a few side hustles to earn some level of income to offset the gap. Hmmmm, maybe I can start a blog? 🙂 I’m 47 years old now, looking to retire in the next couple of years, so we have some time to figure it out. We can also look into ways to tap our tax-sheltered account before age 59 1/2, ideally using methods that don’t impose the 10% penalty. Some options include:

• Building a Roth Ladder
• Take SEPP through 72(t)
• Use the Rule of 55
• Just pay the 10% penalty

Subscribe to get email updates when we post new content and please discuss further in the comments below or let me know if you have any questions or other feedback.

~$Retirement Nerd🤓

Disclaimer: I am not a financial planner and content on this site is meant to provide food for thought, not professional advice. I share my experiences to show what worked so far and what didn’t, YMMV. Please consult your financial advisor or tax professional as needed.