Stop trying to use logic in dealing with things like this. It doesn't matter that no reasonable person could blame Obama for the (idiotic) thoughts of his former pastor. But what matters is that a LOT of people will only hear "Obama's pastor says 'God damn America'" and that will be enough to hurt him.
Dismissing this as illogical is only burying yourself politically. I certainly hope Obama isn't so stupid as to act like this won't be a big deal. If he does then this will be his Swift Boat moment.