This year marks the 5oth anniversary of Buffett and his investment partners’ decision to take control of Berkshire Hathaway. In this year’s letter to shareholders, Buffett takes a stroll down memory lane and recalls how he took over Berkshire Hathaway, which at the time was a struggling textile manufacturer based in Bedford, Mass. Along with that story, Buffett shares what he says were two of his biggest business and investment mistakes, one of which he claims cost him and his investors $100 billion.

The first mistake Buffett mentions is, surprisingly, buying Berkshire. Buffett calls the actions that led up to his purchase of Berkshire a “monumentally stupid decision.” He says he knew Berkshire was a troubled company and that he had only bought into it because he thought it was cheap. Buffett expected to make a quick profit and get out. And he almost did.

But when, in May 1964, Berkshire CEO Seabury Stanton offered Buffett slightly less than what he had promised to repurchase his stake in the company, the 34-year old investor bristled, even though the price Stanton offered would have given Buffett a 52% return for his two-year investment. Instead, Buffett launched a takeover campaign. By the following May, Buffett and his partners had taken control of the company.

It was a bad move.

Berkshire continued to struggle. And Buffett threw good money after bad trying to turnaround the company’s textile mills. He eventually closed Berkshire’s operations two decades later.

But Buffett claims his biggest mistake came two years later. In early 1967, he bought the business that would eventually be the seed of Berkshire’s insurance, investment, and, eventually, conglomerate empire. Buffett bought and merged it into Berkshire, the publicly traded company that he still runs today.

Buffett says if he had bought the insurance business through his investment partnership hedge fund, which he still ran at the time, he and his investors would have captured all of the investment gains he has created over the past 50 years. Instead, he has shared those gains with the public shareholders of Berkshire Hathaway.

But it’s hard to consider this a mistake that Buffett truly regrets. It’s odd to imagine what Buffett would be like now or, indeed, what the world would be like if the legendary investor had remained a hedge fund manager and not the head of what became the most successful publicly traded investment partnership in history, into which anyone, who had the money, could buy into and become rich. And there likely wouldn’t have been 50 years of investment letters and Buffett wisdom for all to read. This year’s letter, like the others, is great.

There are plenty of other gems in the special section of this year’s letter titled, “Berkshire – Past, Present and Future,” particularly if you are a Buffett fan. And if not for Buffett’s $100 billion mistake, there would probably be a lot fewer of those.

Watch more business news from Fortune: