I would say that it depends on the devices you expect your audience to use. Currently, Safari and some iOS browsers don't work great with WebRTC, but it's the best approach to go.

You can solve your problem either by making a peer-to-peer (FullMesh) approach (which might work, since you're gonna use audio only; video wouldn't work well for more than 4 people though); or, you can implant a server that mixes the audio streams and delivers the mixed output to everyone else.

Approach 1) Peer-to-peer (FullMesh) - Pros: easier to implement, no server required - Cons: requires more resources from Client devices (~N² edges/WebRTC connections are required)

Approach 2) Mixing Audio Streams on a Server - Pros: Scales and performs better (N edges/WebRTC connections are required) - Cons: requires a server + additional implementation effort/research

Hope it helps!