I think if the ball hits a hand when it's between the ball and the "offenders" body it shouldn't be a handball (i.e, if it was going to hit another part of his body anyway). But if their hands are out and they stop the ball from going past them using their hand - whether intentional or not - it should be a handball.
You can say it's harsh all you like, but so is sending someone off if they accidentally bring someone down as last man. The point is when you have a set of rules they shouldn't be left open to interpretation.