The players losing any respect they had for him is an issue. As would be the media circus that would follow him around for every press conference and interview.
He'll be banned, sacked for gross misconduct and Saints can move on with a new manager who hasn't tarnished his and the club's reputation.