Economists at Goldman Sachs and Morgan Stanley have certainly noticed. They cut their 2011 U.S. growth forecasts again this weekend, blaming high energy prices for soft spending and warning that months of mushy consumer numbers – brought to you by the next round of government belt-tightening – lie ahead.

“Slower growth is likely to keep the unemployment rate somewhat higher than we previously thought,” economist Jan Hatzius writes.

Goldman and Morgan both expect the jobless rate to tick down just 3 tenths of a percentage point between now and year-end, to 8.7%. And Goldman actually raised its forecast for joblessness at the end of next year to 8.25% from 8% previously. Anyone who believes this is going to count as enough progress to satisfy Ben Bernanke and send the Fed to the exit should have his head examined.

Why are the supposed bulls on the economy getting so glum? Start with a downturn in recent indicators. Weak numbers in recent weeks include a sharp drop in the Institute for Supply Management’s nonmanufacturing survey, along with a rise in first-time jobless claims and a disappointing household employment report, suggest the economy is not going to expand this year or next at the clip economists had foreseen, Hatzius writes.

Part of that unhappy household report was the revelation that the unemployment number many people track – the so-called U6 line, which lumps the jobless together with those who would like to work more but can’t find a willing employer – actually turned higher in April after six monthly declines (see chart, above).

Meanwhile, fully a quarter of teenagers remain unemployed. That’s not a trend that bodes well for coming years.

So Goldman, which came into the year trumpeting its bullish turn and forecasting annual inflation-adjusted growth as high as 4% in the next few years, has had to pull in its horns again. After slashing its first-quarter outlook last month, Goldman on Friday cut its 2011 growth projection to 2.7% from 2.9% and its 2012 projection to 3.2% from 3.8%.

And in what looks an awful lot like wishful thinking, the reduced forecasts at both Goldman and Morgan Stanley depend on high oil prices – which plunged last week after a long run-up — giving back more of their gains. That’s hardly impossible, but is it something we can count on?

To listen to the guys who were selling the 4% growth story not too long ago, it is.

“If the collapse in prices seen over the past couple of trading sessions is sustained, this would provide some meaningful support to the consumer,” Morgan Stanley’s David Greenlaw writes. Goldman economists, Hatzius writes, “expect GDP growth to remain above trend, and to accelerate in late 2012 as the effects of the rise in oil prices begin to fade.”

And if they don’t fade? Don’t worry, there’s plenty of time for another downgrade.

Also on Fortune.com:

• QE3 on the horizon
• The rise of the permanent temporary worker
• Why inflation hawks are still grounded

Follow me on Twitter @ColinCBarr.