The biggest mistake I think many libertarians make is in thinking of Government as some kind of external entity that serves only to oppress The People. But one of the great ideas from The Enligtenment is that it is The People who are sovereign and thus are the ones who establish the Government. The People ARE The Government. So if The Government becomes oppressive it is because The People WANT it to be oppressive.
As I like to say, Goverenment Happens. What matters is not whether we have Government but how we choose to live with it and make it work. Many libertarians seem to just want the Government to go away and leave them the fuck alone. I am sympathetic to the sentiment. I agree with it many times myself. But it just isn't realistic.
Governments are just one form of Institution that The People create to help manage their affairs. Another is Business. Business is an economic Institution formed by a partnership of People who want to achieve a common economic goal (Make Money Fast!). But Business suffers from the same flaw as any other Institution: once created, it can take on a life of its own.
The real danger to society comes when the Institutions we create to manage our affairs begin to act as if The People are subservient to them (Autocracy) rather than the other way around (Democracy).
That's why I am a Democrat. Democrats believe that The People are the ultimate source and purpose for the existence of Institutions (both Government and Business). The People are Sovereign and, as such, The People have the right to regulate those Institutions in whatever way they deem necessary. Voting is just one means of doing it. Delegating authority to representatives is another. In Government this delegation produces a Republic. In Business this delegation produces a Corporation(*).
Republicans, on the other hand, lose sight of where Sovereignty comes from and approach Institutions as if it were the delegates who have the sovereignty. Thus they call themselves Republicans (supporters of The Republic) instead of Democrats (supporters of The People).
Democrats want The People to regulate Institutions. Republicans want Institutions to regulate The People.
(* - Corporations are Businesses that are given a measure of independent existence entirely by action of The People (articles of incorporation). So anytime anyone says the Government, the manifestation of the will of The People, doesn't have the right to regulate Corporations, I call bullshit.)