Nope, for several reasons.
One, that letter is from Conservatives Against Racism For Equality, meaning Baker would have been up poop creek without a paddle if he ignored it, so he's simply being a politician here. There's no condemnation in what he says - it's simply "I fear" nonsense covering his own arse, and you've seen the Tory response from Schapps this morning to slap him down. Politically, small amounts of activists will be ignored in favour of the wider electorate; Tory advisers will be correctly stating that the Tory leadership shouldn't and won't be playing this game.
Two, again, if you have to explain what the "modern" meaning of taking the knee is, the gesture fails. Automatically. That person goes to great lengths to try and tell people what taking the knee means, simply saying it's "wrong" that it's linked to Kaepernick/BLM - connotation simply doesn't work that way. You don't get to decide what the gesture means simply because you don't want it to mean something else.
Three, the letter actually backs up what I've said in one sentence - "racism shouldn't be about left and right, it should be about right and wrong." Absolutely. The knee fails to make it that. So change the gesture - because it fails, absolutely.