We shall never know, of course. Hart doesn't make anything of it at the time. The guidance from FIFA is:
“interfering with an opponent” means preventing an opponent from
playing or being able to play the ball by clearly obstructing the opponent’s line of vision or movements or making a gesture or movement which, in the opinion of the referee, deceives or distracts an opponent.
It must have been the referee's decision although why we needed to go to his assistant I don't know.