(Anthony, response to you is below Michael's response to consolidate messages). Apologies in advance for the long reply - but love the discussion.
Hey Michael,
It's not so much the number of clients that is problematic but rather the difficulty of capacity planning when operations become increasingly more expensive. It presses towards building a smaller number of very large FS instances, over a smaller number of medium sized ones that you could get away with for audio + network traffic. The system I'm building is multi-tenant, and there's application code written around distributing conferences across FS hosts. But say an FS host has 80% CPU, it's hard to know in advance if a conference I'm about to place there will turn into a 10 person video chat or a 2 person audio chat. With an MCU approach, there's a very real possibility that if 10 people join I exhaust CPU and run into issues. Whereas with an SFU, there's much less risk of any one conference pushing over that edge, because that 20% can still handle a considerable number of clients. It just makes the capacity planning and distribution process easier when a typical video conference consumes a very small number of CPU resources.
As far as the second comment - I agree. Certain devices may be limited in their processing power. But mobile devices are also limited in screen real state, so there may not be an MCU layout that works well for a combination of both mobile and desktop devices. With an SFU approach, you could choose to receive all streams on the mobile device but only decode 2 or 3 of them (at the application's discretion), and lay them out in a way that is optimal for that device. It wastes network bandwidth, but preserves experience. You could do similar with a canvas per caller, but then you're even further increasing the cost on the server.
In any case I'm not intending to argue that SFU is a better approach generally (or argue at all for that matter), just trying to better understand the required additions to FS to make it happen. With the addition of mod_verto and full native WebRTC support, having an SFU capability would allow FS to cater to the FaceTime/Hangouts/Skype types of applications with highly customized (and hopefully optimized) experiences.
Anthony,
I agree that what I'm referring to would be little more than a replication of what Jitsi does, but the "value add" would be having it baked directly into mod_conference with no need to interface with another product. I'm finding it hard to rationalize how I would get Jitsi Video Bridge working with Freeswitch in a way that doesn't cause more problems than it solves - and FS caters to far more use cases independently than Jitsi can. To clarify, I'm not asking anyone here to contribute the code - rather just looking for any pointers on what it might take. If SFU were to be added as an exclusive OR with the current canvas muxing, is this a PR that the FS would accept? Or is it a requirement that it be able to operate in a hybrid MCU + SFU mode? I'm still not quite sure how useful that would be, though - except in the cases of better supporting mobile (for which there are SFU workarounds), or older devices that don't do multi-stream (seems unlikely that these use cases would want SFU in the first place though).
Sorry again if I'm annoying anyone! I'm really just looking for the dialog/hoping to learn more.
Best,
Colin