To illustrate his point, Paul describes a hypothetical situation in which the whole church is together for public worship and everyone speaks in tongues (supposedly without interpretation).
The phrase outsiders or unbelievers
broadly refers to all those who are not part of the church. The Christian worship service has always been a public service, open also for non-Christians to attend.
It is not difficult to imagine what unbelievers would think if they would find everyone in the church speaking in different languages. At Pentecost the apostles themselves were accused of drunkenness (Acts 2:13).
23 Si donc, dans une assemblée de l'Eglise entière, tous parlent en langues, et qu'il entre de simples auditeurs ou des non-croyants, ne diront-ils pas que vous êtes fous?