Norwich didn't spent anywhere near £5m. Most of their transfers came from the lower leagues.
Saints in the past two years have spent close to £5m on transfers/loans/agent fees/signing on fees. Now they are in a league which gives you £4m extra in TV money each year, larger attendances, more sponsorship, more solidarity payments etc etc.