It's the old saints supporters are racist line- until you remember that our scapegoat for a number of years was Jermaine Wright. Yes I know he is vaguely black but not dark enough for the racist hate I would imagine.
That is a completely different debate to the one raised by Whelan. He didn't say certain social groups do better than others did he. Any "seeing what I am trying to do" is just your usual paranoia or possibly a bit of guilt. You can decide which.