When the economy isn't doing well, it's mostly the lower and middle classes who end up with tighter budgets, eg less money to spend on entertainment expenses and things like coins. So prices on the lower and middle end coins stagnate. By contrast, the entertainment budgets of the rich aren't effected nearly as much, so higher/high/very high end coins will retain their value or increase still.
There's also the matter of, if an individual/family in a crunch needs to part with some or all of their collection, it is very often the lower and middle end coins that they sell/put into the marketplace. Supply and demand means the prices for this more common stuff goes down. The rich with the top coins will only part with them if they absolutely must because they fear at auction they won't fetch as much as they could. This has been a recent theme in recent years, with both collectors and dealers claiming that there's no good "fresh" coins available to buy.