I think that's a little harsh. The problem with a balanced trade agreement is that it compels legislation from all parties. It can never stay 'just trade', because enforcing a trade agreement requires things like agreeing on standards that ultimately exceed the boundaries of 'just trade'. So, very strictly speaking, you're correct to assert that he is in error on that front.
I also don't think it's incorrect to assert that the EU has moved into areas that have nothing to do with trade, or that some of those policies are broadly unpopular among the UK electorate. The question boils down to: are the benefits of cooperation worth more than the price of complying with policies the electorate doesn't like? That's a question of preferences over outcomes, and how distasteful giving up sovereignty on those issues is. There really isn't a 'right' or 'wrong' answer on that one, which is why the whole problem was so contentious.
The mudslinging doesn't help things.