It definitely does contribute to becoming a winning team in the long run. Obviously if you're someone like Wigan, it's not realistically going to make you suddenly into contenders for the league, but if you're trying to build long term success, you absolutely need the winning mentality.
Just look at Arsenal now, they don't have that experience of winning trophies and it showed last week. City, despite being behind them in the league, had that big game mentality and experience.
So often you see big managers talk about the importance of this and really needing tangible signs of progress for fans and players and evidence that the team can win things. It breeds confidence, even if it's subconscious. How often do you see teams lose their first few semifinals and finals before they crack it and then go on to have sustained success. This City team for example, lost a few finals before they finally become a trophy machine.
Also, let's be realistic, we aren't winning the league anytime soon (possibly ever), so why would you not want to win an actual trophy?