The S&P 500 is down 20% year-to-date, and with bonds simultaneously experiencing their worst year in history, there’s been nowhere to hide. After more than a decade of strong returns following the Great Financial Crisis of 2008, most investors are unaccustomed to this kind of carnage.

Between 2009 and 2021, the S&P 500’s average annual return was a hefty 16.4%, according to New York University. But don’t expect anything like those gains in 2023.

Investment banks’ average price target for the S&P 500 next year is roughly 4,000, implying stocks will rise just 4%. When Fortune collected forecasts from a wider selection of Wall Street economists and analysts, that figure rose to 4,150, or a 8% gain in 2023. Still, that’s nothing like what most investors are used to. 

While investment banks believe the S&P 500 will end 2023 just slightly above where it stands today, Wall Street’s growing consensus is that the path to get there will be choppy.

“Our main message to investors is to be cautious. The Fed is trying to engineer a soft economic landing that in our view has a high likelihood of failing and causing a recession in 2023,” James Demmert, chief investment officer at Main Street Research, told Fortune. “Stock indexes are vulnerable at current levels.” 

Watch out for a rough start to 2023

Across Wall Street, investment strategists warn that stocks are in for a rough start to 2023 as the Fed enters the final stages of its inflation fight. Year-over-year inflation, as measured by the consumer price index (CPI), has fallen sharply from its 9.1% June peak, hitting 7.1% last month.

But that didn’t stop the Fed from raising interest rates for the seventh time this year on Wednesday. Even as critics argue that rate hikes are driving the U.S. economy toward recession, Fed Chairman Jerome Powell said this week that he intends to “stay the course until the job is done.”

“Worse pain would come from a failure to raise rates high enough and from us allowing inflation to become entrenched in the economy,” he said.

Against this backdrop, the stock market will likely struggle in the near term.

Morgan Stanley’s chief investment officer, Michael Wilson, said that the S&P 500 could drop to between 3,000 and 3,300 in the first quarter—or as much as 25% below current levels. Earnings will take a hit as companies cope with rising borrowing costs and slowing economic growth, he said, while arguing that investors aren’t yet anticipating the drop.

“Markets were ignoring the risk of a more hawkish Fed a year ago; the market now appears to be ignoring earnings risk,” he wrote in a Monday research note, adding that the “risk/reward” proposition of investing in the S&P looks “very unattractive” at the moment.

Scott Ladner, chief investment officer at Horizon Investments, told Fortune that he expects a 10% decline in the S&P 500’s earnings-per-share over the coming months as the Fed’s interest rate hikes slow the economy and hit corporate profits.

“We haven’t seen earnings get taken down yet,” he explained. “We are going into a slower growth period, maybe a recession, and you just don’t come through recessions with earnings that don’t fall some.”

A second half comeback story?

While Wall Street’s consensus forecasts call for stocks to drop in the first quarter of the year, it’s a different story after that.

“We expect that when the market bottoms—perhaps in the first quarter of the new year—we will start a new bull market,” Main Street Research’s Demmert said. “Although there may be further weakness in the first part of 2023, we expect 2023 to end with stock prices significantly higher than today’s levels.” 

Horizon Investments’ Ladner said he also expects markets to go through some “pain” in the first quarter, but after that, investors can expect solid returns.

“We think that the back half of the year could be pretty juicy,” he said. “Many things that were pretty severe headwinds in 2022 may end up being tailwinds in 2023.” 

Ladner argued that the major issues that hurt stocks this year—inflation, China’s strict COVID zero policies, and the war in Ukraine—will likely be resolved or improved in 2023, providing a boost for markets.

The second half comeback thesis has become common on Wall Street in recent months. Economists, investment banks, and hedge funders are all warnings that stocks—and the economy—will likely struggle in the first half of the year and then rebound.

Morgan Stanley’s Wilson argues that after falling to 3,000, the S&P 500 could rise to 3,900 by the end of next year as inflation sinks rapidly, leading the Fed to pause its rate hikes as early as January.

While it may seem like a second half comeback story flies in the face of the consistent recession predictions from Wall Street this year, Ladner pointed out that stocks can actually perform well during recessions.

“Most of the retail people don’t get that, but it’s the lead up to the recession and the first part of the recession that usually gives markets problems,” he said.

Ladner also argued that forecasts for stocks to be flat for a decade due to persistent inflation, deglobalization, and rising interest rates strike him as “betting against innovation.”

“That’s a bad idea,” he said. “Just historically, that’s always been a bad idea in this country. So that’s not a bet we’re taking.”