It's human nature that people will turn Cameron's "reluctance" into a personal attack but, as you say, the opposition will always come across better in these presidential-esque debates because it's always easier to sell change than it is to defend the status quo (hence why Nick "promise the world" Clegg came out top last time).
Take the Scottish separation referendum for example...."Yes, let's create utopia" is always going to have a more positive vibe to it than "No, lets stay the same, it's worked OK for 300 years" regardless of the merits of each sides's policies.
Blair didn't want the debates (and succeeded in avoiding them) and neither did Brown (who didn't). And you can bet your bottom dollar that Miliband won't be clamouring for them in 5 years time, should he get into number 10 on 7th May (*shudders*)