When dealing with Christians we also read that we are NOT to handle legal disputes in the courts of the unbelievers (
I Corinthians 6:8). The fact that civil marriage would require a Christian couple to go to court to dissolve their union should they be faced with divorce is a reason that many Christians choose not to have a civil marriage.
In addition, Paul describes how a Christian should deal with abandonment by an unbelieving spouse...
15 But if the unbelieving depart, let him depart. A brother or a sister is not under bondage in such cases: but God hath called us to peace..