Home values over the long run tend to rise just slightly faster than inflation, making it a worse investment than, say, investing in the stock market.

But these are not normal times. Though the economic recovery is six years in the making and home prices have recovered to their pre-bubble norms, the real estate market has changed radically—mostly for the worse.

Even creditworthy borrowers are having difficulty securing a mortgage to buy a home. Real estate data firm Zillow released a report this week showing that buying a home makes financial sense, on average, after living in the same home for just two years. This varies depending on location. For instance, in expensive areas like Washington, D.C. and Los Angeles, buying a home starts to make sense after you’ve lived there for 4.2 and 5 years, respectively.

Even as housing prices have recovered, rents in many areas have been climbing even faster, tilting the math in these cities toward buying versus renting. But as the housing finance industry continues to absorb the effects of the housing bust and the subsequent regulatory response, lenders are still gun shy about getting involved in residential real estate. Laurie Goodman, an economist at the Urban Institute, displayed this in a presentation given last month. Here’s one of her slides, which shows the evolution of the median home price, and the homes that median Americans can afford to buy, if he devotes a third of his income to mortgage payments, fees, and taxes:

As you can see, the gap between what the average Americans can afford and the median sales price is much larger than it had been prior to the housing bubble. Another chart shows that this is largely because lenders are being extremely picky about whom they lend to:

Here we see that folks with FICO scores above 720 are receiving the majority of new loans, whereas in 2000, before the real estate market was close to bubble territory, the distribution was much more even. Goodman argues that the reason for this shift is that the mortgage finance system still hasn’t recovered from post-crisis uncertainty.

Government-backed Fannie Mae and Freddie Mac are still serving as the anchor of mortgage finance in this country. Most loans are guaranteed by the housing giants, and lenders are scared of selling anything but the most flawless loans to them out of fear that the agencies will force lenders to buy back the loans if they fail. At the same time, heightened regulatory scrutiny has forced mortgage originators to spend far more time on each mortgage they underwrite, to the point that the average lender is processing one-sixth of the mortgages per month they were doing back in 2001.

Meanwhile, any urgency in Washington to reform housing finance and limit Fannie Mae and Freddie Mac’s role in this space has evaporated. A recent report from Standard and Poors argued, “We expect Fannie and Freddie to be around for the foreseeable future in their current form, and we think the government will support them.”

Under its previous leader, Ed DeMarco, Fannie and Freddie moved to raise the fees they charge to insure mortgages in an effort to encourage private mortgage guarantors to enter the market. But under new leadership, which is facing the reality that the status quo of these government-backed entities may continue for some time, further fee increases have been delayed. Meanwhile, according to S&P, “the enterprises must continue to pursue initiatives to expand what loan originators consider an eligible borrower, including evaluating alternative credit scoring models, assessing impediments to credit access, and clarifying operational standards with other market participants.”

There are signs that these efforts are starting to work. For instance, the share of first-time home buyer loans Fannie and Freddie are purchasing inched up last year. But, as S&P puts it, “The FHFA has pushed for more clarity in the market, mainly by making it easier for originators to be free of representation and warranty claims and trying to increase standardized documentation, but those efforts have probably only marginally helped.”

Since mortgage finance reform is unlikely in the near future, Fannie and Freddie are going to have to work much harder to reassure originators that they won’t get stuck buying back mortgages that default. Obviously, we don’t want to return to the lax standards of the housing bubble, nor should taxpayers be put on the hook for poorly underwritten loans. But we should be able to find a happy medium between the reckless housing bubble years and today, when few but the most creditworthy can secure a home loan.