Colossians 2 says it the most clearly. In a nutshell, just as circumcision was a mark of being entered into the old Abrahamic covenant, baptism similarly serves a similar purpose. It is the mark of the new covenant.
11 In whom also ye are circumcised with the circumcision made without hands, in putting off the body of the sins of the flesh by the circumcision of Christ: 12 Buried with him in baptism, wherein also ye are risen with him through the faith of the operation of God, who hath raised him from the dead. 13 And you, being dead in your sins and the uncircumcision of your flesh, hath he quickened together with him, having forgiven you all trespasses;
Instead of circumcision of the flesh, this is a "circumcision made without hands" (verse 11), or a spiritual circumcision, of the heart.
Instead of putting off of the male foreskin, baptism serves as a "
putting off of the body of the sins of the flesh". This of course is in line with other scriptures that tell us baptism is for "
the remission of sins" (
Acts 2:38) or for "
washing away of sins" (
Acts 22:16)