Paul is not necessarily saying that the believers in Galatia have left Christ. He is warning them that their desire to keep the law as a means of justification is at odds with faith in Christ, and they will fall from grace if they continue on their course.1
4 Vous êtes séparés de Christ, vous tous qui cherchez la justification dans la loi; vous êtes déchus de la grâce.