The drops in U.S. scores go beyond the pandemic and cut across income, geographic and racial divides, new data shows.