The Quaker couple who originally introduced me to the idea of marrying without involving the government pointed out a passage of Scripture to me that they believe most Christians don't consider when they think about marriage. They point to
I Corinthians 6...
I Corinthians 6:1-8 (ESV)
When one of you has a grievance against another, does he dare go to law before the unrighteous instead of the saints? 2 Or do you not know that the saints will judge the world? And if the world is to be judged by you, are you incompetent to try trivial cases? 3 Do you not know that we are to judge angels? How much more, then, matters pertaining to this life! 4 So if you have such cases, why do you lay them before those who have no standing in the church? 5 I say this to your shame. Can it be that there is no one among you wise enough to settle a dispute between the brothers, 6 but brother goes to law against brother, and that before unbelievers? 7 To have lawsuits at all with one another is already a defeat for you. Why not rather suffer wrong? Why not rather be defrauded? 8 But you yourselves wrong and defraud—even your own brothers!
They believe that by entering a civil marriage, a couple is legally obligating themselves to the civil court system and its statutes. Thus issues relating to divorce or other disputes must go through a secular civil court, putting them to open shame before an unbelieving society. They feel that as believers they are able to judge in themselves, and with the assistance of their local meeting, the issues they might face in their union should they be confronted with troubled waters or the need for separation. They'd rather suffer wrong and even be defrauded before allowing their union to become legally entangled in the civil court system.
And so, for this reason they feel the purist form of Christian marriage is government-free.