David Miliband told MPs: "Given that this was a very sophisticated operation, in which high-quality forgeries were made, the Government judges it highly likely that the forgeries were made by a state intelligence service.
"Taking this together with other inquiries and the link to Israel established by the Serious Organised Crime Agency (Soca), we have concluded there are compelling reasons to believe Israel was responsible for the misuse of the British passports."
Mr Miliband also confirmed a member of the Israeli embassy was being "withdrawn" from the UK at his request.
The Foreign Secretary told Sky News: "It is important that Israel understands that the engagement we seek with them is on the basis of trust and transparency, which has been violated in this case.
"It is important that the government in Britain stands up for the people in Britain."
Gordon Brown's spokesman told reporters: "The Prime Minister was consulted on this decision and supports it."
The spokesman said the diplomat was being given two weeks to leave the country.
Israel's foreign ministry said it regretted the UK's decision.
The Israeli ambassador to London, Ron Prosor, said Israel was "disappointed by the decision of the British government," but affirmed his commitment to a relationship "of mutual importance."
Mossad, Israel's secret service, has widely been blamed for the murder of Mahmoud al Mabhouh in January.
Dubai police investigating the case revealed some members of the suspected hit squad had travelled on British identities.
Their passports appeared to have been copied from real documents belonging to at least 12 Britons living in Israel.
Mr Miliband added that the misuse of British passports was "intolerable" and said the Foreign Office would amend its travel advice to "alert other British nationals to the risks that their passports might be misused in the same way".
Hamas said it welcomed the British Government's decision to expel a diplomat and accused Israel of being a "practising terrorist state".
Sky's foreign affairs editor Tim Marshall said the Government was not taking action over the assassination, but the duplicated documents.
"The Government believes they have compelling evidence that those passports were cloned, probably by some arm of the Israeli state.
"This is not a direct accusation by the UK that they have implicated Israel in the assassination.
"What Britain is saying is, they have evidence that the passports were cloned and they take it so seriously they are going to expel an Israeli diplomat."
Knesset member Aryeh Eldad was not impressed.
"I think [the] British are behaving hypocritically and I don't want to offend dogs on this issue, since some dogs are utterly loyal, who are they to judge us on the war on terror?"
He told Sky News Israel should expel a British diplomat in response to the UK's action.
At the time of the passports revelation, Mr Miliband called the abuse of documents "outrageous" and demanded Tel Aviv co-operate with the UK's own investigation.
Detectives from the Soca were sent to Israel to discover how the British passports came to be used.
Dubai officials said they were "99% certain" that Mossad agents were behind the hit, but Israel has refused to confirm or deny this.
Al Mabhouh founded Hamas' military wing, which has been responsible for hundreds of deadly attacks targeting Israelis since the 1980s.
The terrorist, who was said to be 50, was killed in his luxury hotel room on January 20 - a day after he arrived in the city.
Others allegedly involved in the assassination used fake Irish, French and Australian travel documents, Dubai police said.
Interpol have issued arrest warrants for 27 suspects wanted in connection with the Dubai hit.
