These special interest groups all compete for dollars, just the same way politicians compete for votes. If they don't serve their constituency their funding will decrease.
The problem isn't special interests, per se. The problem is that certain special interest groups have the ability to raise far more money than others, and this ability has nothing to do with their political popularity. If a company makes its money from hauling trash, what part of its ability to raise money from trash hauling would make you want it to be allowed to have a large influence on our political system? The goal of campaign finance reform was to limit that influence and equalize to some degree the opportunity that people had to influence campaigns through monetary donations. Our political tradition is "one man/woman one vote" - having certain actors spend much more money than others warps that tradition, since it gives people with excess cash an oversize platform to spread their message.
It's one thing if a politician accumulates campaign money through many small donations, but if that money is only coming from a few sources, the opportunity is there for a candidate to win election but feel more loyalty to just the few special interests who paid for his campaign. Even this is less of a problem if those special interests are broad based groups that at least represent large numbers of people (unions, animal rights groups, Audubon Society, NRA, pro-life groups, whatever), but if it's just a few large corporations then the notion of democratic rule becomes seriously compromised. Yes, even the most compromised candidate in the world still has to win elections, but it's widely recognized that in modern campaigns you need money to win elections, and a compromised candidate with money will usually beat a clean candidate who doesn't have money. It's not a sure thing, but if taking the money is going to lead to victory more than 50% of the time (and I'd say evidence suggest it does far more often than that), you'll find most candidates are going to take the money. If money is a necessary part of the political system, then the only way to ensure that people have relatively equal access to / ability to support & influence candidates is to limit the size of donations.
Now you could say that everyone has an equal ability to make money, and if you want to have the same ability to influence politics as Wal-Mart or Google, just go get rich. This is trite. From a practical standpoint, the vast majority of people will never be able to be that wealthy. It's unhealthy for the one man one vote concept to have so much political power in the hands of just a few people (since we've established, money allows the purchase of political power - just see John Corzine, for example - sure his money was unable to preserve his seat as NJ governor, but it's equally true that it bought him four years in the job in the first place). The rich can't completely dominate - a populist cause or candidate will occasionally come along and derail things, a really poor candidate (see Corzine) will fail if he's a spectacular failure - but they can certainly control things most of the time (and they do).
About the only argument for this that I can see is simply that successful people (the wealthy, wealthy corporations) should be in charge of the country, so what's the problem? To some degree I have to agree with this - I would say that is why the system has more or less worked up to this point. If the abject poor had as much influence a the rich currently do now (and the rich had as little influence as the abject poor), it would probably be an unmitigated disaster. Still, there are certainly some potential pitfalls to allowing the wealthy to have so much influence. At the end of they day, the wealthy are probably going to look to preserve their wealth first, even at the expense of what's good for the country as a whole (this is true of every economic group, but if a few hundred thousand wealthy people are looking out for their interests at the expense of the other 299 million of us, that's a lot more problematic than if the 150 million or so of the middle class are looking out for their own interests). This may be a problem we simply have to deal with in the interest of preserving free speech (that's what the SCOTUS is arguing, I suppose), but it's disingenuous to claim it doesn't exist.
Ce n'est pas une pipe. C'est une signature.