Even though the average American’s 401(k) balance rose 2% since the first quarter of the year, the average account was still down 2.5% for the three months ending in June compared to a year ago, according to a Fidelity analysis of 14.2 million people through the end of June. The average Individual Retirement Account, or IRA, balance also fell by 7% compared to the same quarter in 2015, based on 8.2 million IRA accounts.

Those figures come in an atmosphere of low interest rates, which depress bond yields, and a relatively flat S&P 500 over the 12 months ending June. 401(k)s are often a mix of stock, bonds, and/or cash.

But those swings in the market over the past year often caused by macroeconomic shocks such as the crash of the Shanghai Composite in 2015 and the Brexit referendum, also relegated some investors to the sidelines.

“Most retirement savers are accustomed to market volatility, but the swings in the second quarter were especially dramatic, including a 600-point drop followed by a nearly 800-point increase,” Doug Fisher, Fidelity’s senior vice president for workplace investing, said in a statement.

Though just 8% of Fidelity’s consumers made changes to their accounts during that 12 month period.

According to Fidelity’s director of investments, Meghan Murphy, the main reason why the average account balance feel was because of “the influx of millennials into our system, so they come in with zero assets, and thousands of baby boomers retiring everyday and dipping into their accounts, are definitely impacting our averages.”

That comes as 32% of Americans told Fidelity earlier this year that their retirement savings are not on track to match the life they have planned in retirement.

Meanwhile, the U.S. is already facing a $1 trillion pension shortfall due to an aging population and tight state budgets.