Personally I would politely refuse and if anyone asked say that empty gesturing is not for me. If anyone objected to that I would point out that in removing my choice, you’ve essentially made the gesture completely meaningless.
The problem is many people won’t see it that way. If a PL player decided not to take the knee tonight for example, he’d be crucified in the press, SM and by commentators, who would completely ignore the fact that he’d already taken the knee three times previously.