It's because, i'd imagine that those fans of the little teams in the league, and in the non league, well their fans would stay in and watch the premier league, even if 100-200 were shaved off the gate each week, over the course of 23 home games that's potentially 4,600 people, which for some teams, is a full fixture of revenue, or in some cases, 2 games. It'd cripple the football pyramid.
Edit: As for Premier League teams, teams like Fulham already take rubbish followings, that'd get even worse and even Everton would lose a good couple of thousand each week imo, Everton Vs. Stoke, where's the appeal in paying £40 when it's on the box for free?